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ABSTRACT 

T. is report comprises a detailed evaluation of three 
IBM microcomputer-based school information management systems: 
Student Information and Records System (SIRS) by Management 
Information Group, The School System (TSS) by Columbia Computing 
Services, and Computer Educational Management Accounting System 
(CEMAS) by Computerlib. These three systems were evaluated against 
six major factors, each defined by a detailed and comprehensive set 
of criteria: product scope and function, ease of use, technical 
considerations, support and services, produce qualifications, and 
vendor. All evaluations were conducted in schools usir.g real and full 
school data, and all key system capabilities were tested as they 
related to database creation and maintenance, prescheduling, 
scheduling, transition to operational status (and semester turnover), 
attendance recording and reporting, progress recording and reporting, 
report generation, aad utility functions. Each product evaluation 
describes the testing environment and conditions, l.sts evaluation 
results and observations, and summarises the strengths and weaknesses 
of the system. Results show that one system, The School System by 
Columbia Computing Services, can effectively meet the n&eds of both 
junior and senior high schools. Six appendixes are included: the 
general questionnaire from which the criteria were derived, the 
interview guide and detailed checklist, the detailed scoring 
comparison form, SIRS user group member list, recent product 
developments, and a full re-simulation of the relative suitability of 
the three systems. (TB) 
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1.0 INTRODUCTION 



Recent rapid advances in the computer technology and related fields have 
greatly increased the spectrum of opportunities for the application of 
computers. WWile increasing in power and performance, computers have also 
" ecome more affordable and easier to use. Increasingly, educational 
administrators are seeking to apply the technology to the administration of 
schools. Many tasks which were once .:onsid a red addressable only by large 
centralized mainframe computers can now be addressed by microcomputers. An 
example ot such tasks is organization tor instruction. School administrators 
are becoming increasingly interested in the local application of computer 
technology to school information management. 

Among the computer based applications which exist for school administrators 
today are School Information Management Systems (SIMS) with a particular focus 
on student related information. These systems may be microcomputer or 
minicomputer based and, typically, incorporate four major nodules which 
address school records, student scheduling, student attendance and marks or 
progress reporting. Usually, there is a high degree of integration between 
the modules which means, for example, that duplicate data bases are not 
required. In most cases, the cost of these software systems belies their 
complexity. Four thousand dollars buys multi-megabytes of software 
opportunity. In all cases, it is safe to assume that the cost of the software 
system itself will be the least impacting factor in any decision to apply it. 

The purpose of t.ie work which is reported on here was to evaluate the 
comparative suitability of two microcomputer based SIMS for use at the senior 
high school level. This endeavour was one component of a more global 
investigation of SIMS alternatives for high school use. In particular, 
Edmonton Public Schools and Alberta Education jointly funded the investigation 
of minicomputer based approaches to school information management as well. 
This initiative will be the focus of another report to be released in the near 
future. All investigations (of both mini and microcomputer based systems) 
were performed according to a thorough and objective evaluation process which 
was developed specifically for the purpose. The approach to evaluation is 
described in detail in a report entitled Selection Criteria for Integrated 
School Information Management Systems (available from Alberta Education). 

In view of the extemely high general level of interest in this area, the scope 
of the project was widened (in two ways). Firstly, three systems were 
evaluated rather than two and secondly, the systems were evaluated for their 
suitability to junior high schools rather than just to the senior high 
schools. 

The systems evaluated were 

o Student Information and Records System (SIRS) by Management Information 
Group 

o The School System (TSS) by Columbia Computing Services 
o Computer Educational Management Accounting System (CEMAS) by 
Computerlib 
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The evaluation of CEMAS began in October 1983 and was completed in October 

1984. SIRS was evaluated in two phases - the first phase was between April 
1984 and June 1°84 while the second phase was between Oct 1984 and January 

1985. The School System was evaluated between October 1984 and January 1985. 

All the systems were evaluated on IBM Microcomputers. 
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2.0 APPROACH TO EVALUATION 



2.1 Evaluation Criteria 

The three systems under investigation were evaluated against six major 
factors^ These major evaluation factors were: 



Product 3cope and Function (what does it do and how well does it do it) 

Ease of Use (user friendliness) 

Technical Considerations (system design, structure, operation etc) 

Support and Services (after sales service) 



o Product Qualifications (product credibility, history, etc. 



o 



Vendor (who stands behind the product) 



Each of the six major evaluation factors was defined by a detailed and 
comprehensive set of criteria. Information gained from consultations with 
schools was ppramount in the de/elopment of the criteria. The criteria were 
developed through a six step process as outlined below: 

Step 1 A General Questionnaire (see Appendix 1), Interview Guide and 

Detailed Checklist (see Appendix 2) were developed for the gathering 
of information from the schools. These documents were developed 
using information gained through prior, extensive contact with 
schools in general, through the experiences of Information Services 
staff, and with a working knowledge of the characteristics of 
currently available systems. The general questionnaire was designed 
to determine which features and characteristics a SIMS should include 
and, in many cases, their relative importance. Where measures cf the 
relative importance of a criterion or characteristic were required, 
the questionnaire featured a simple four point "must, "important", 
"optional" and "not required" scale for respondents to check. 

Step 2 Eighteen district schools were identified as a representative sample 
through which detailed school information management needs and 
requirements would be confirmed. These schools were carefully chosen 
to reflect many of the key variables such as school level, size, 
programs, organization and operational style. 

Step 3 The General Questionnaire was sent to the 18 identified schools 

togethe- with a statement of iLs purpose and instructions for its 
completion. Participating schools were requested to give careful 
consideration to t'leir responses to the questionnaire and to prepare 
for a follow-up interview. The questionnaire also allowed 
participants to respond to needs and requirements not specifically 
identified in the survey. 



ERLC 



I0> 



St After allowing ample time for the completion of the questionnaire, 

follow-up interviews were conducted at each school using the 
Interview Guide and Detailed Checklist inferred to previously. The 
purpose of this step was to clarify and confirm responses relative to 
the questionnaire. A key reason for the two stage information 
gathering process (questionnaire followed by the interview) was to 
allow the schools to first respond without external influence of any 
kind. 

Step 5 Information gathered through the administration of the questionnaiLe 
and subsequent interviews was compiled and analyzed and used to 
determine the relative importance of selection criteria items. 
Particular attention was paid to the comments of participating 
schools since this sometimes led to the inclusion of additional 
criteria items which might otherwise have been missed. 

Step 6 Simple qualitative and quantitative analysis of the questionnaire, 

its findings, and the resulcs of the interviews led to the definition 
of the detailed criteria as well as to the determination of weighting 
factors. The Detailed Evaluation (o^ Selection) Criteria in tabular 
form and a description of the column entries are shown in the 
following pages. 
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PROCUCT 

score * 

FUNCTION 



EVALUATION 
FACTOR 



CMTEriA ITEMS 
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90 POOL RECORDS 
Pre-Reglstratlon/Enrollment 
Create student record 

- school student I.D. 

- last nam 

- middle name 

- first name 

- birthdate 

- current grade 

- sex 

- feeder school 

- home address 

Registration confirmation notice 
Feeder school confirmation notice 

TOTAL ?r/-Registrati /Enrollment 

Detailed Data Item s 

Student information 

- school student l.D. 

- District s'.udent I.D. 

- Alberta Education student I.D. 

- last name 

- Ed d die name 

- first name 

- birthdate 

- current grade 

- sex 

- feeder school 

- home address 

- telephone number 



WEICKT 
(V) 



15 



SCORE 
(S) 



2D 



25 



WEIGHTED SCORE 
(V X S) 



MAX V7 SCT*£ 
(W X * m% ) 



vr scoRf/nyc vr score 
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EVALUATION 
FACTOR 


1 

CRITERIA I TEWS 


WEIGHT 
(W) 


SCORE 
(S) 


! 

WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

< w x w 


1 

WT SCORE/MAX WT SCORE 




- emergency contact 

- name 

- telephone 

- entry information 

- entry date 

- registration code 

- withdrawal code 

- previous schools (2) 

- homeroom instruction 

- counsellor 

- parent/guardian Information (up to A) 

- name 

- address 

- telephone (home and business) 

- relationship 

- occupation 

- locker Information 

- number 

- combination 

- student indebted'- hp 

- religious denomination 

- program type 

- number of credits earned 

- this school 

- other schools 

- academic history 

- travel information 

- method 

- distance 

- bus pass information 

- parking informs t ion 

- driver** licence 

- licence plate 

- parking space 

- medical information 

- disabilities/behaviours 

- medications 

- allergies 













1 EVMJJATION 1 


1 1 t,i\ LA I 1 l/u 


LfFinfT 

we. t on i 

(v) 


(S) 


(W X S) 


may i/T ernpr 

(w x s mx ) 


\n c udrk/max wt scorf 




- date of last medical 

- physician information 

- health care number 

- departure information 

- date 

- reason 

- minliaun of 6 user defined fields 














Instructor Information 


5 












- instructor code 

- name 

- address 

- telephone 

- social insurance number 

- language of instruction 

- certificate number 

- courses taught 

- adnlmuft of o user defined fields 














Course information 


15 


1 










- course code (5 character alpha-numeric) 

- description 

- pre-and co-requisites (minimum of U) 

- most handle**and*7 , *or"situation 

- course type 

- language of instruction 

- course accreditation 

- credit value (2 digits) 

- pass/fall mark 

- g**ade 














TOTAL Detailed Data Items 


45 
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0? 
0) 
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ft 
1 



EVALUATION 
FACTOR 
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CRITERIA ITEMS 



Reports/Inquiries 

All reports and inquiries should be avail- 
able for all or a specified range of 
records, in various sort orders. 

- class lists 

- homeroom lists 

- student name labels 

- student address labels 

- parent address labels 

- student I.D* carda 

- student data (alphabetical or numerical 
order) 

- parent data (alp* abetical or numerical 
order) 

- Instructor data (alphabetical or numer- 
ical order) 

- course data 

- student phone list 

- student name list 

- student grade list 

- feeder school list 

- loclcer information list 

- student population by instruction type 

- fee sheets 

The system should allow production of 
user-defined reports/inquiries using 
available data* 

TOTAL Reports/ Inquiries 

TOTAL SCHOOL REGQKDS 



wEicirr 
(w) 



25 



SCORE 
(S) 



WEIGHTED SCORE 
(V X S) 



MAX WT SCORE 

(w x s MX ) 



WT SCORF/HAX WT SCORE 



5? 



Q- 

to 
M 

to 
< 

M 
C 



O 
3 

O 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



SOfEDULDC 
Detsiled Data It 

- Course code 

- Course section 



Manual scheduling (Arena Scheduling) 

Pre- scheduling 

Course Requests 

manual entry 
automated entry 

- allow student to specify mandatory/ 
compulsory courses, 

- preferred courses, preferred 
alternatives, etc. 

- allow student to specify preferred 
sec. Ion, semester » or instructor 



Edit and validation of course requests 

- checking of pre- and co-requisites In 
the current students 1 requests as well 
as history files 

• capability to override pre- and co- 
requt sites 

- capability to complete pre-requlslte 
checking for students from other 
District schools. 

P re-scheduling reports 

- potential conflict matrix — for all 
or a specified range of courses. 
Additional selection criteria may be 



WEIGHT 

'0 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX UT SCORE 
(V X S ) 



UT SCORE/MAX UT SCORE 
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EVALUATION 



CRITERIA IT!>« 



WE1CKT 
(V) 



based on the nunber of requests or the 
number of sections, 

- course tally 

- students with no requests 

- student course request list 

- mln/max request list 

- mln/max credit list 
• verification tickets 

- arena scheduling labels 

- stjdents missing compulsory courses 

- students requesting specific course or 
group of courses 

Master schedule builder 



Capability to build a master schedule 

annually 

automatically 
Capability of handling a variety of 
Scheduling units 

- full year 

- semester 

- trimester 

- quartermeater 

- 6 week unit 

- any combination of the above 

User defined timetable rotation/tumble 
Flexible number of periods per day 
Capability to specify exclusive male or 
female sections 

Capability to maintain current and future 
year/semester master schedules 
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rOORE 
,S) 



WEIGHTED SO0R£ 
(WX S) 



"AX WP SCORE 



(V 



VT SOORTVMX WT SCORE 



m 
in 

M 

in 
< 

0) 
C 



O 
3 



n 



EVALUATION 
FACTOR 



CTkTFRU ITEMS 



Scheduling Process 

User defined scheduling sequence 

- low grades first 

- high grades first 

- A to t 
u to A 

Unscheduilng of iw-shovs/vithdrawals 

Scheduling of Individual student or small 

groups of students 

Capability to rase, all students or 

partially scheduled students 

Capability to lock scheduling assignments 

for all students or a group of students 

Restart capability 

Course weighting/semester balancing 

(ensure even course load for students) 

Blocking of courses 

Section balancing 

Class balancing (males-f<\.iales) 

Capability to keep scheduling open after 

school start while starting to use the 

attendance nodul. 

Scheduling Reports/Inqulrlet 

- student timetables — grid and list, 
format 

- instructor ciicetables — grid and list 
format 

- oom timetables — grid and list format 

- master schedule 

- student scheduling conflicts 

- students partially scheduled 

- una* signed time 



OA 



WEIGHT 
(W) 



5_ 
6 



8 
8~ 



10 



score 

(S) 



WEIGHTED SCORE 
(W X S) 



MAX Vf SCORE 
(W X S^> 



wr score/kax vr sore 
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EVALUATION 
FAC^R 


CRITERIA ITEMS 


VEICirr 
(W) 


SCORE 
(S) 


WEICKTEO SCORE 
(W X S) 


MAX VT SCORE 

(w x S-BX ) 


WT SCORE/MAX VT SCORE 




Junior High Scheduling Requirements 














Homeroom grouping for core subjects 


9 












Capability of scheduling any course In 
any combination and number of time 
periods 


10 












TOTAL SCHEDULING 


200 












STOCKS f ATTENDANCE 














En^ry of Attendance Data 














manual entry 


5 












automated entry 


9 












Multiple user-defined absence types 


8 












Capability to record attendance data at 
various Intervals 


is, 












- dally 

- twice per day 

- period by period 

- subject by subject 














Attendance history 


8 












- at least ten days detail 
• cumulative totals 














Atic;idance reports/Inquiries 


10 












- student by class 

- student by subject 

- student by period 






■ 

i 







ft> 



m 

cx» 

to 

M 

CO 

0? 

M 

C 
0? 



o 

3 

o 

1 
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nilTERIA 



wEiarr 

(w) 



SCORE 
(S) 



- homeroom attendance 

- daily summary 

- weekly summary 

- monthly summary 

- multiple absence 

- capability to produce unexcused 
absence report for the current day 
within 30 minutes 

- the system should allow user defined 
reports/ inquiries using available data 

TOTAL ATTENDANCE 



STUDENT MARKS 

Entry of marks data 

manual 
automated 

Marks data 

- minimum of 4 term marks plus final mark 

- letter or percentage grades 

Student Exams 
Exam timetable builder 

- automated 

- manual 

Exam Reports/Inquiries 

- potential exam conflict matrix 

- exam schedules 



50 



10 



WEIGHTED SCORE 
(W X S) 



MAX UT SCORE 

(w x Smax ) 



WT SCORE/MAX UT SCORE 



CL 

to 

M 

X 
c 



o 

3 

o 
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EVALUATION 
FACTOR 


CK'TERIA 1TO4S 


(W) 


MX/Rfc 

(S) 


ucirtirrn cpvauc 
VrtlCflTtD 

(w x S) 


MAY LPT C/VlDr 

MAX WT SCuRE 

(W x s^) 


w| oiAJKr./ riAA WT SCORE 




R* ports/Inquiries 


10 












proof list 
report cards 

- marks data 

- final mark, calculated according to 

- user-defined form 

- attendance data 

- class averages 

- honour lists 

- potential failure lists 

- pradiuition lief 














TOTAL STUDENT MARKS 


40 












UTILITY FUNTTTffTC 














Backup/ Res tore 


1 ? 
1/ 












Secur 1 ty / Con t rol s 


0 

o 












TOTAL uTTLITT PUNCTIORS 




























GRAND TOTAL, PRODUCT SCOPE AND PUNCTION 




400 






















EASE OP 


- flexibility 


60 










DSE 


- nodular, table driven 

- help facilities 

— menu driven 














GRAND TOTAL, EASE OF OSE 




60 


































i 



EVALUATION 
FACTOR 



TECHNICAL 
CONSIDERATIONS 



CRITERIA ITWS 



WEICHT 
(W) 



SUPPORT A 
SERVICES 
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hardware 

system software environment 

- operating system 

- utilities 

- database management /system 
internals/files 

- networking capabilities 

- user hooks 

- modularity of the system 



CRAKO TOTAL, TECHNICAL CONSIDERATIONS 

local versus where /how far 
package support and services 
- software support, custom 
modifications 

documentation 

- user guide , application system, 
procedural, operations guide, 
file layouts 

training 

- applications system, operational 
(DP), availability schedule, format 
location, prerequisites 

implementation 

- training 

- initialization (conversion^ lie set 
up, output forms) 

- implementation plan 



GRAND TOTAL, SUPPORT 4 SERVICES 



80 



80 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



70 



70 



MAX VT SCORE 
(W X S^) 



VT SCORE/MAX VT SCORE 
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EVALUATION 
FACTOR 


CRITERIA ITOTC 


(V) 


oWKfc 
(S) 


WEIGHTED SCORE 
(V X S) 


MAX WT SCORE 

(w x % mx ) 


WT SCORE/MAX WT SCORE 


PRODUCT 
QUALIFICATIONS 




80 












- package background 

- reliability 

- current development status 

- nuuber of installations 

- product development plans 

- release concept, portability, 
verticality 














CRAIfD TOTAL, PRODUCT QUALIFICATIONS 




80 
























VENDOR 




70 














- Corporate information 

- background and history 

- financial performance 

- employee base 

- Market volatility and vendor stability 

- References 

- Contractual Terms 

- BAlncenance 

- warranty 

- ownership rights 

- discount structure/price limit 














GRAND TOTAL, VENDOR 




70 







































The extreme left hand column of the tables shows the major evaluation 
factors. The column immediately to the right of this displays the criteria 
items • Major criteria items a r e underlined. Below each major criteria item 
is a list of detailed criteria. The detailed criteria are of two types - 
those against which the systems under evaluation will be scored and those 
which are to provide context for the scoring process. Criteria provided for 
context purposes are identified by a preceding hyphen. Those criteria against 
which systems were scored can be identified by the presence of an entry in the 
column marked WEIGHT (weighting factor). 

The column entries for the Criteria Tables are defined as follows. 



Evaluation Factor 



Criteria Item 



Weight 



identifies a key area of evaluation and 
the beginning of a detailed criteria list 
for that particular factor. 

identifies a feature, process or attribute 
associated with the factor. The Criteria 
item column also contains supplementary 
entries intended to p^vide an evaluator 
with a more complete spective on a 
particular criteria item being 
evaluated. Supplementary entries, which 
are identified by a preceding hyphen, do 
not have a weight assigned to them. 

is a measure of the relative importance of 
a criteria item to the user. Summing of 
weighting factors (or weights) gives a 
broad perspective of the relative 
importance of major areas or modules 
within the context of the entire 
evaluation. Weights are assignable at the 
discretion of the user. 



Score 



Weighted Score 



Maxima Weighted Score 



is a measure of how well a given criteria 
is met by a particular alternative. It is 
suggested that scores be assigned on a 
simple 0-10 scale (or user defined 
equivalent). Only those items which have 
weighting factors should be scored. 

this column entry is the product of the 
weight and the score and is a measure of 
how well the needs of a user are met on 
that particular item, area or module, 

is the product of the weight and the 
maximum possible score. This would be the 
weighted score which implies a perfect fit 
to the needs of the user on a particular 
criteria item, set thereof, factor, etc. 
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Weighted Score/Max Weighted Score - this ratio gives a proportional measure of 

how well user needs are met on a parti- 
cular item, set thereof, factor, etc. 

For those evaluators who may wish to compare raw and weighted scores across 
product alternatives, a Detailed Scoring Comparison Form was also developed 
(see Appendix 3). This particular form is identical in format to the Detailed 
Evaluation Criteria Form but contains only those items which were scorable 
(i.e. it does not include context re lated items ) . 

2*2 Evaluat 4 Method 

All evaluations were conducted in schools using real and full school data. 
Wherever possible, live or current school data vas used. When this was not 
possible, data associated with a known reference point was used. While the 
actual testing was peformed by programmer/systems analysts, school 
administrators were maximally involved with the key decisions and judgements 
which guided the evaluations. This was one of the most important reasons why 
the evaluations were conducted in the schools. All key system capabilities 
were tested particularly as they related to: 

o Data base creation and maintenance 

o Pre-scheduling 

o Scheduling 

o Transition to operational status (and semester turnover) 

o Attendance recording and reporting 

o Progress recording and reporting 

o Report generation 

o Utility functions 

It is not possible to list all evaluation considerations for all criteria in 
this report - some key performance considerations, however, were the quality 
of results achieved, completion times for major procedures and reports and 
inquiry response times. 

During the course of the evaluations, each system was scored against each of 
the evaluation criteria using a zero to ten point scale. Scores were assigned 
as o\erall measures of "performance" against the criteria taking into account 
all considerations believed to be relevant by the evaluation team. 

For example, consider the scheduling process. Both the timing and the quality 
of the result are critical evaluation considerations. Competitive systems 
might receive equivalently low scores if, while one produces a high quality 
result (e.g. high % students completely scheduled) in a very long time frame, 
the other produces a low quality result in a very short time frame. 

In isolation, the mere presence of a particular feature, the sheer speed with 
which a process could be completed or the high quality of a particular result 
are not necessarily consistent with the awarding of high scores. 

Testing and evaluation was supervised by three different project leaders on 
the Distributed Systems Team (of Edmonton Public Schools 1 Information 
Services). All software systems were evaluated in IBM microcomputer 
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environments. Every attempt was made to maximize objectivity. For examp 1 ^, 
each system was t/aluated by more than one project leader and frequent 
meetings were held to ensure cross referencing and the sharing of ideas and 
experiences. Despite this, of course, it is reasonable to expect some 
subjectivity to exist characteristic of the particular evaluator. 

3.0 OVERVIEW OF SYSTEMS EVALUATED 

Student Information and Records System (SIRS), developed by a small Alberta 
based company, underwent initial pilot testing in Fedwater School in the 
County of Sturgeon. The system was initially developed as a pupil records 
system which included Alberta Education interfaces and with links to a 
financial information management system. The system evolved with the addition 
of a scheduler, the development of which was heavily based on the requirements 
of Redwater School. Close contact was maintained with at least one Edmonton 
Public School during this particular phase of SIRS' development. 

SIRS was originally developed in RM COBOL to run on the NCR Tower Minicomputer 
and the multi-user ALTOS microcomputer. The system was converted to run on 
the IBM PC XT microcomputer. 

Many of the programs which constitute SIRS have been re-written, particularly 
those relating to scheduling, to add more function and increase the speed of 
operation. SIRS is a system which integrates the basic functions relating to 
school records, student scheduling, attendance and progress tracking and 
reporting. A SIRS user's group was recently formed to provide input for 
future product enhancement and development. At the time that this report was 
produced, there were seven known installations of SIRS (see Appendix 4) all of 
which are in Alberta. SIRS, on the ALTOS and NCR Tower computers at least, is 
a multi-user system. 

The School System was developed by Columbia Computer Services, a Vancouver 
based company. Columbia is a company which focuses exclusively on the 
education market and has almost two decades of design and development 
experience in student information management systems. The Company has a large 
North American customer base and has for a number of years (since 1968) 
offered services to schools (particularly scheduling) through a mainTrame 
based service bureau approach (time sharing). Hundreds of North American 
schools are known to have subscribed to this service. ColumDia is now in the 
process of phasing out its mainframe based services to customers in favour of 
a microcomputer based product which it has developed, called The School 
System. The School System features an integrated approach to school records, 
student scheduling, attendance and progress tracking and reporting. This 
multi-user system is written in the C language and was developed specifically 
to run on the IBM family of microcomputers. To date, The School Svstem has 
been installed in more than 200 schools. 

Computer Educational Management Accounting System was developed by a small 
Toronto based company called Computerlib. This product evolved from a product 
called EMAS which was developed to be marketed on the IBM System 34 Mini- 
computer. Initially, CEMAS ran on the Xerox microcomputer (and also, the 
Radio Shack microcomputer) and was subsequently converted, with some re-design 
and re-development, to run on the IBM PC and PC/XT. CEMAS, which was 
developed in a mixed language environment (including the PASCAL and C 
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Languages), was the first microcomputer based product that we evaluated. 
The IBM PC version of CEMAS was an immature product at the time of its 
evaluation by the Team. During the period cf our involvement with CEMAS we 
became aware of only two or three other attempts to install the product 
operationally. We are not aware of the current number of installations of 
this product. As with SIRS and TSS, CEMAS addresses the four major functions 
of schools records, student scheduling, attendance and progress tracking and 
reporting. At the time of the evaluation, CEMAS did not offer multi-user 
capability. 

Since completing the formal evaluations of the three microcomputer based SIMS 
in February 1985, there have been a number of product announcements and system 
enhancements. Appendix 5 provides an overview of major new developments that 
are known to us . 

4 »° PRODUCT EVALUATIONS - SENIOR HIGH SCHOOL PERSPECTIVE 

As stated previously, all products were evaluated in IBM microcomputer 
environments and at school sites. Though multi-user capability is considered 
to be important (particularly for the larger schools), testing of this 
capability was not included within the scope of these evaluations. From the 
product scope and function perspective, systems were evaluated in a stand- 
alone fashion. Two of the products offered multi-user capability at the time 
of evaluation notably SIRS and TSS. The developers of CEMAS indicated that 
multi-user capability was definitely included within the scope of their 
product development plans. It should be noted that scores of zero were used 
to indicate total absence of a capability or feature. 

4.1 Evaluation of SIRS 

^•l.l Testing Environment and Conditions 

The initial phase of testing was conducted using Jasper Place School - a 
school of about 1750 students which offers a wide variety of programs 
including vocational. An IBM PC XT complemented by a 10 Mb expansion unit and 
an Okidata Microline 84 printer were used for this phase. The second phase 
was conducted at Eastglen School - a school of 775 students, Second phase 
testing was conducted, using 2 different IBM personal computers, specifically, 
an IBM PC XT/370 with 10 Mb expansion unit and an IBM PC AT with a built in 
20Mb hard disk. Two different printers, the Okidata Microline 84 and an 
Epson MX 80 were used. 

It was a requirement that all systems evaluated be able to run on IBM 
microcomputers. With this condition clearly defined, MIG converted its 
NCR/ Altos based system to run on the IBM microcomputer in order that it be 
included among the products to be evaluated. MIG advised the evaluation team 
that not all of the many programs which constituted Si AS were converted and 
that we might also expect performance degradation (particularly speed of 
execution) as a general consequence of the conversion to IBM format. 

4.1.2 Evaluation Results and Observations 

The following tables show the outcome of the quantitative evaluation of SIRS 
against the detailed criteria. 
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EVALUATION 
FACTOR 



CRITERIA ITEM C 



PRODUCT 
SCOPE & 
FUNCTION 



SCHOOL RECORDS 
Pre-Reglstratlcn/Enrollment 
Create student record 

- school student I.D. 

- last name 

- middle name 

- first name 

- blrthdate 

- current grade 

- sex 

- feeder school 

- home address 

Registration confirmation notice 
Feeder school confirmation notice 



TOTAL Pre-Regiatratlon/EnrollMit 



Detailed Data Items 



Student information 



- school student I.D. 

- District student I.D. 

- Alberta Education student I.D. 

• last name 

- middle name 

- first n.^me 

- birthdate 

- current grade 

• sex 

- feeder school 

- home address 

- telephone number 




WEIGHT 
(W) 



15 



20 



25 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 

< W X S m*») 
max 



WT SCORE/MAX WT SCORE 



120 



10/30 



126 



200 



.63 



200 



41 



42 
ERIC 1 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


J WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

< w x W 


WT SCORE/MAX WT SCORE 




- emergency contact 

- name 

- telephone 

- entr/ information 

- entry date 

- registration code 

- withdrawal code 

- previous schools (2) 

- homeroom instruction 

- counsellor 

- parent/guardian information (up to 4) 

- name 

- address 

- telephone (home and business) 

- relationship 

- occupation 

- locker information 

- number 

- combination 

- student indebtedness 

- religious denomination 

- program type 

- number of credits earned 

- this school 

- other schools 

- academic history 

- travel information 

- method 

- distance 

- bus pass information 

- parking information 

- driver's licence 

- licence plate 

- parking space 

- medical information 

- disabilities/behaviours 

- medications 

- allergies 








! 





EVALUATION 
FACTOR 



CRITERIA ITEMS 



WEIGHT 
(W) 



SCORE 
(S) 



WEIGHTED SCC RE 
(W X S) 



MAX WT SCORE 



WT SCCRE/MAX WT SCORE 



- date of last medical 

- physician information 

- health care number 

- departure information 

- date 

- reason 

- minimum of 6 user defined fields 



ERIC 



Instructor Information 

- instructor code 

- name 

- address 

- telephone 

- social insurance number 

- language of instruction 

- certificate number 

- courses taught 

- minimum of 6 user defined fields 
Course information 

- course code (5 character alpha-numeric) 

- description 

- pre -and co-requisites (minimum of 4) 

- must handle"and M /"or"situation 

- course type 

- language of instruction 

- course accreditation 

- credit value (2 digits) 

- pass/fail mark 

- grade 

TOTAL Detailed Data I tew 



35 



15 



120 



45 



23/30 



35S 



450 



.788 



45 



EVALUATION 
FACTOR 



48 
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CRITERIA ITEMS 



Reports/ Inquiries 

All reports and inquiries should be avail- 
able for all or a specified range of 
records, in various sort orders* 

- class lists 

- homeroom lists 

- student name labels 

- student address labels 

- parent address labels 

- student I*D. cards 

- student data (alphabetical or numerical 
order) 

- parent data (alphabetical or numerical 
order) 

- instructor data (alphabetical or numer- 
ical order) 

- course data 

- student phone list 

- sttident name list 

- student grade lis' 

- feeder school list 

- locker information list 

- student population by instruction type 

- fee sheets 

The system should allow production of 
user-defined reports/inquiries using 
available data* 

TOTAL Reports /Inquiries 

TOTAL SCHOOL RECORDS 



WEIGHT 
(W) 



25 



25 



90 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT S r RE 



WT SCORE/MAX WT SCORE 



150 



39/70 



150 



631 



250 



900 



.60 



•70 



1 



EVALUATION 
FACTOR 
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CRITERIA ITEMS 



SCHEDULING 

Detailed Data Items 

- Course code 

- Course section 

Manual scheduling (Arena Scheduling) 

Pre-scheduling 

Course Requests 

manual entry 
automated entry 

- allow student to specify mandatory/ 
compulsory courses, 

- preferred courses, preferred 
alternatives, etc* 

- allow student to specify preferred 
section, semester, or instructor 

Edit and validation of course requests 

- checking of pre- and co-requisites in 
the current students' lequests as well 
as history files 

- capability to override pre- and co- 
requisites 

- capability to complete pre-requisite 
checking for students from other 
District schools* 

Pre-scheduling reports 

- potential conflict matrix — for all 
or a specified range of courses. 
Additional selection criteria may be 



WEIGHT 
(W) 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



63 



35 



36 



63 



81 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



19 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



WEIGHT 
(W) 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



based on the number of requests or the 
number of sections. 

- course tally 

- students with no requests 

- student course request list 

- min/tnax request list 

- nrtn/max credit list 

- verification tickets 

~ arena scheduling labels 

- students missing compulsory courses 

- students requesting specific course or 
group of courses 
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Master schedule builder 

Capability to build a master schedule 

manually 

automatically 
Capability of handling «i variety of 
Scheduling units 

- full year 

- semester 

- trimester 

- quartermester 

- 6 week unit 

- any combination of the above 

User defined timetable rotation/ tumble 
Flexible number of periods per day 
Capability to specify exclusive male or 
female sections 

Capability to maintain current and future 
year/semester master schedules 



6 


7 


9 


0 


9 


8 



42 



72 



10 


8 


10 


8 


5 


8 


8 


10 



80 
80 



40 



80 



EVALUATION 
FACTOR 
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CRITERIA ITEMS 



Scheduling Process 

User defined scheduling sequence 

- low grades first 

- high grades first 

- A to Z 

- Z to A 

Unscheduling of no-shows/withdrawals 

Scheduling of individual student or small 

groups of students 

Capability to reset all students or 

partially scheduled students 

Capability to lock scheduling assignments 

for all students or a group of students 

Restart capability 

Course weighting/semester balancing 

(ensure even course load for students) 

Blocking of courses 

Section balancing 

Class balancing (males-females) 

Capability to keep scheduling open after 

school start while starting to use the 

attendance module 

Scheduling Reports/ Inquiries 

- student timetables — grid and list 
format 

- instructor timetables — grid and list 
format 

- room timetables — grid and list format 

- master schedule 

- student scheduling conflicts 

- student? partially scheduled 

- unas signed time 



52 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



42 



45 



56 
56 
72 
32 



90 



MAX WT SCORE 
(W X W 



WT SCORE/MAX WT SCORE 



-53 — 1 



EVALUATION 
FACiDP 



CRITERIA ITEMS 



T untor High Scheduling £veqnlren»c.it3 

Homeroom grouping for core subjects 
Capability of scheduling any course in 
any combination and number of time 
periods 

TOTAL SCHEDULING 



STUDENT ATTENDANCE 



Entry of Attendance Data 

manual entry 
automated entry 

Multiple user-defined absence types 

Capability to record attendance data at 
various intervals 



- daily 

- twice j;er day 

- period by period 

- subject by subject 

Attendance history 



- at least .<;n days detail 

- cummulative t "als 

Attendance reports/inquiries 



- student by class 

- student by subject 

- student by period 



WEIGH! 
(W) 



181 



10 



10 



SCORE 



WEIGHTED SCORE 
(V X 



144/240 



1065 



45 



64 



90 



56 



70 



MAX WT SCORE 
^ X W 



WT SCOrlE/MAX WT SJORE 



1810 



.58 



55 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHl 
(W) 


SCURE 
(S) 


WEIGHTED SCORE 
(W X S) 


1 

MAX WT SCORE 

(W X S ) 
v max 7 


WT SCORE/MAX WT SCORE 




- homeroom attendance 

- daily summary 

- weekly summary 

- monthly suninary 

- multiple absence 

- capability to produce unexcused 
absence report for the current day 
within 30 minutes 

- the system should allow user defined 
reports/inquiries using available data 














TOTAL ATKNDANCK 


50 


40/60 


325 


500 


•65 




SiiJDENT MARKS 














Entry of marks data 














manual 


5 


3 


40 








automated 


9 


0 


0 








Marks data 


10 


8 


80 








- minimum of 4 term marks plus final -nark 

- letter or percentage grades 














Student Exams 


6 


0 


0 








Exam timetable builder 














- automated 

- manual 














Exam Reports/Inquiries 














- potential exam conflict matrix 

- exam schedules 
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57 





EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(:>) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

< w x w 


WT SCORE/MAX WT SCORE 






Reports/Inquiries 


10 


8 


80 












proof Hoc 
report card? 

- marks data 

- final mark, calculated according to 
use^-defined formula attenandance data 

- class averages 

- honour lists 

- potential failure lists 

- graduation list 


















TOTAL STUDENT MARKS 


40 


24/50 


200 


400 




.5 






UTILITY FUNCTIONS 


















Backup/Restore 


12 


8 


96 








lo 
O 




Security/Controls 


8 


2 


16 












TOTAL UTILITY FUNCTIONS 


20 


10/20 


112 


200 




.56 
























GRAND TOTAL, PRODUCT SCOPE AND FUNCTION 


381 




257/440 




2333 




3810 




.6123 






EASE OP 


- flexibility 


60 


6 


360 










USE 


- nodular, table driven 

- help facilities 

- nen i driven 


















GRAWD TOTAL, EASE OF USE 


60 




6 






t 

360 I 




600 




.6 






















Etyc 



















EVALUATION 
FACTOR 



CRITERIA ITEMS 



HT 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 

(W X S ) 
v max' 



WT SCORE/MAX WT SCORE 



TCOWICAL 

CONSIDERATIONS 



SUPPORT & 
SERVICES 



hardv na 

system software environment 

- operating system 

- utilities 

- database management /system 
internals/files 

- networking capabilitiec 

- user hooks 

- modularity of the system 



GRAND TOTAL, TECHNICAL CONS IDC? ATIONS 



local versus where/how far 
package support and services 

- software support, custom 
modifications 

documentation 

- user guide, application sv^tem, 
procedural, operations guide, 
file layouts 

training 

- applications system, operational 
(DP), availability schedule, format, 
a v nation, prerequisites 

implementation 

- training 

- initialization (conversion , file set- 
up, output forms) 

- implementation plan 



GRAND TOTAL, SUPPORT & SERVICES 



80 



480 



80 



480 



800 



.6 



70 



490 



70 



490 



"eric 



700 



• 7 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



PRODUCT 
QUALIFICATIONS 



ho 



VENDOR 



02 
ERJC 



- package background 

- reliability 

- current development status 

- number of installations 

- product development plans 

- release concept, portability, 
verticality 



GRAND TOTAL, PRODUCT QUALIFICATIONS 



- Corporate information 

- background and history 

- financial performance 

- employee base 

- Market volatility and vendor stability 

- References 

- Contractual Terms 

- maintenance 

- warranty 

- ownership rights 

- discount structure/price limit 



GRAND TTAL, VENDOR 



WEIGHT 
(W) 



80 



80 



70 



70 



SCORE 
(S) 



WEICHTED SCORE 
(W X S) 



480 



480 



490 



490 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



00 



] 



Observations 



The following comments and observations are offered in support of the 
quantitative evaluation of SIRS. 

(A) Scope and Function 



School Records: Positive pointr include a 15 character identity field 

which facilitates the link to Edmonton Public School's 
mainframe based pupil records system; family linking 
capabilities which means only one contact for several 
students within a family; proper pre- and co-requisite 
checking with the capability to override this when 
necessary; good Alberta Education field for statutory 
reporting; existence of a link between student and 
course fees to an MIG financial system which includes 
student fee invoices* 

Negative points include absence of user definable 
fields ( considered to be very important); absence of 
some less important fields; absence of registration 
and feeder school confirmation; use of artificial 
numerical codes for student requests, probably 
introduced to accomplish course grouping or linking; 
use of numeric codes in areas such as the instructor 
file; no user defined reports. 

Scheduling: The scheduling function is capable of working with 

both semestered and non-semestered versions of the 
same course using the same student request. In 
addition, good tally reports and conflict matrices are 
available; flexible class placement is possible 
anywhere within the period by day matrix; there is a 
capability to specify, for each request, preferred 
semester and alternate course information; quite 
acceptable class balancing is achieved with results 
obtained comparable to those achieved using the 
mainframe and minicomputers. 

On the negative side, use of :he artificial numeric 
rode as a course request rather than the course code 
itself tends to complicate the scheduling process; 
there is a necessity to run several time consuming 
edit reports before each scheduling run; it is 
necessary to completely close scheduling nefore 
starting attendance and other general school 
functions: there is a long processing time when 
sheduling large schools; necessary information is not 
printed with partial schedules which could be used to 
resolve conflicts. 
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Attendance: The Attendance module is fairly acceptable with 

flexible user defined codes which may be classified as 
"accumulating" types and "non-accumulating" types for 
attendance letter generation; there is the capability 
o* generating 8 different letters to parents which are 
automatically prepared when attendance problems pass 
certain level?; detailed period and class attendance 
is intained for the whole year; generation of 
attendance collection forms is good as are facilities 
for data entry. 

Negative points include too much detail on absences in 
most reports; codes for absence reasons are numeric 
rather than the more meaningful alphabetic codes used 
in other systems; the generation of colleccion 
registers could be slow for a large school especially 
if class lists are to be alphabetic which seems to be 
the norm. 

Student Marks: The marks system is adequate with a structure 

consistent with Alberta Education requirements and a 
reflection of the current move towards departmental 
exams; the fomat of student report data is consistent 
with current EPSB reports; the required final marks 
and attendance summary js automatically transferred to 
the academic history segment for use in pre-requisite 
checking. 

Some poot features include the absence of automatic 
marks entry; all marks are stored as percentage scores 
even if awarded as pass/fail or A,B,C, etc; there are 
no user definable marks storage nor calculation 
modules which can store a lar3e number of intermediate 
marks irequired to calculate a report card mark. 

Utility Functions: Backup and Restore are menu driven and are considered 

normal and acceptable. Security v~as not present on 
the evaluated system with the exception of access to 
some technical system set».;p parameter files 

(3) Ease of Use 

The system is not flexible in the sense of user 
defined fields, user defined reporting, user 
controllable import and export capabilities • ^ne user 
is bound by a menu driven system which in itself does 
not allow much diversion from system defined 
procedures. 

No help facilities are availble in SIRS. 
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(C) Technical Considerations 



The system was designed to r U n op an ALTOS multi-user 
microcomputer, or an 1 CR Tov;er minicomputer, hence the 
performance and overall functionality of the package 
was adversely affected after being converted to run on 
the IBM PC ramily of computers. 

The software was written in RM COBOL and operates 
through semi-interpreted object modules and a run-time 
system. 

There is extra information (JCL type statements on the 
screen) which tends to "clutter" the screen and break 
the fluid movement from screen to screen and menu to 
menu. 

(D) Support and Services 

Positive points include the fact that the company and 
product are local; there is a relatively small 
customer base and the product is still in development 
which means that the vendor is willing to support the 
product well and customize; the vendor is very forth- 
coming with respect to system capabilities and 
explanations as to how functions are performed; there 
is a willingness to write custom programs to allow 
data downloading from central d-" cabases; the vendor 
readily became involved with implementation plans for 
testing purposes and responded rapidly to queries and 
sof tware problems • 

On tho negative side, documentation is very poor, 
basically a collection of screen dumps with 
practically no explanation; only very minimal formal 
training was available - the philosophy seemed to be 
to try to work with uhe system and the company will 
help as problems arise; the small size of the company, 
the limited number of installations jf SIRS and the 
potential impact of staff changes or. product support 
are considered signif icant reasons for concern • 

(E) Product Qualif icationr 

The package was developed for the Alberta market with 
close contact with Alberta Education. Initial 
implementation , testing and direction f r 
modifications and the development of the scheduling 
module were influenced by at least one Edmonton Public 
School. 



Hardware: 



Software: 
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The current development plans are based on ALTOS/NCR 
versions and revolve around: 

- adding function and speed to the scheduling area 
(this is badly needed). 

- interfacing with the NCS SENTRY 3000 form scanner. 

- interfacing to "surveyor" type auto dialing systems. 

The current user base includes seven Alberta 
installations based on information available in 
January 1985 . 

(F) Vendor 

The vendor is a relatively small company with some 
local customers. The company is based in St. Albert 
and developed the system as an adjunct to a financial 
package. The vendor sells hardware as well as 
software and offers a good price structure on the SIRS 
sy stem. 

Unfortunately, the combination of snail size and lack 
of good references works against the vendor. 
Similarly, limited staff and vendor stability are 
minus points • 
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4.1.3 System Performance, Strengths and Weaknesses - SIRS (NJ G) 



Key Performance Indicators 



School Test Site 



Parameter 



IBM PC/XT PC/AT 



astgleti CHS 



Scheduler - Time 
Scheduler Performance 
Scheduler - Expected Perf, 
Timetables 

Conflict Matrix 
Course Tally 
Master Schedule 
Class Lists 

Attendance Registers 
Marks Registers 
Student Registers 



4:10 hours 
0:50 hours 



2:30 hours 
62% 
65% 

7:00 hours (list 
format) 



2:15 hours 

5:00 hours (non- 
alphabetic^ 

- t CO hours 

6:20 hours 

i:00 hoi'rs 



Jasper Place CHS 



Scheduler - Time 
Scheduler - Performance 
Scheduler - Expectec Perf. 



46 hours 19 hours 

85% 85% 
85% 85% 



Jasper Place CHS 
Eastglen CHS 



1846 students 
775 students 



(£11 timings are in hours: minutes) 
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System Strengt t is ; - Good prerequisite checking before and *;ter 

scheduling, hcwever the;e act as a warning only 
and do not affect loading, thus desired 
exceptions can be scheduled, 

- Good fee information held at the student and 
course level for preparation of fee invoices • 

- A convenient link to the mainframe database via 
the 15 character field for the Student. ID. #. 

- LoCdlly developed: thus changes in Alberts 
Education requirements should be more easily 
accommodated • 

- Changes might be more likely to be considered 

- A number of reports specifically designed for 
Albert? Education are oresent. (Possibly not 
a greac advantage to those '/ho report at the 
district level - rather than at the school 
level) 

- Automatic generation of attendance letters (8 
user defined types) based on user defined 
amount of absences • 

- SIRS is known o run in a multiuser (non '• 
PC) environment 3nd has print spooling 
capabilities. 

System Weaknes s es : - Poor performance on IBM equipment (primarily 

speed) • 

- Screen organization i readability not as good 
as other systems. Frequent display of JCL type 
statements and diff'culty in finding the 
appropriate field or m.-nu selection. 

- Lack of automated input on IBM equi praent at 
this point in tiiae . 

~ Use of a 3 digit request code for scheduling 
purposes causes user confusion and tends to 
make the scheduling process slow to edit and 
run. 

- Lack of any user defined fields which wi 11 
result in the need for continuing modifications 
at the vendor level. 



69 
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4.2 Evaluation ol TSS 



4.2 vl Tesring Environment and Co nd itions 

Two separate project teams, each unae a project leader, evaluated The School 
System at a number of senior high schools. In ail cases, IBM FC/AT and IBM 
PC/XT computers were used for the work. 

Team 1: Initial testing was on an IBM PC/XT computer. 

Final testing :o^i place on IBN. PC/AT computers 
on full sets of data for Jasper Place and W.P. 
Wagner schools. 

The configuration of the IBM D C/AT system in each 
case was: 512 kb memory, 20 Mb hard disc, PC DOS 
3.00 operating system. 

All reports were printed on OK I DATA Microline 
printers. 

Team 2: Tr 2 School System was tested at three schools, 2 

Senior High and cne Junior High (see section 6.0 
for details of the Junior High School 
evaluation). At J. Percy Page school, an IBM PC 
connected onto the Davong Multilink network vas 
used with a 5 Mbyte use. volume and 448 kb 
memory* Printing was carried out on a General 
Electric Genicom (300 characters per second) 
printer. 

At Victoria Composite High school, an IBM PC/AT 
was used with 20 Mb hare* disk storage, 512 tbytes 
memory and an OKIDATA 84 printer. 

4.2.2 Evaluation Results and Observations 



The following tables show the outcome of the quantitative evaluation of 
Columbia's "The School System" against the detailed criteria. The results and 
the following observations are a consensus of information from the two teams 
which undertook the evaluations. 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



PRODUCT 
SCOPE & 
FUNCTION 



SCHOOL RECORDS 
Pre-Registration/Enrollroent 
Create student r ecord 

- school student I.D. 

- last name 

- middle name 

- fjrst name 

- birthdat" 

- current grade 

- sex 

- feeder school 

- home address 

Registration con c irroation notice 
Feeder school confirmation notice 



TOTAL Pre^Re^trardWEnroll«ent 

Detailed Data Items 

Student information 



:RJC 



- school student I.D. 

- District student I.D. 

- Alberta Education student I.D, 

- last name 

- middle name 

- first name 

- birthdate 

- current grade 

- sex 

- feeder school 

- home address 

- tel e*>nono number 



WEKiHT 

(<0 



SCORE 
(S) 



15 



20 



13/30 



25 



WEIGHTED SCORE 
(W X S) 



90 



15 



109 



MAX WT SCORE 

(w x Smax ) 



WT SCORE/MAX WT SCORE 



200 



.545 



200 



*10 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 
< W X W 


|wT SCORE/MAX WT SCORE 


73 


- emergency contact 

- name 

- telephone 

- entry information 

- entry date 

- registration code 

- withdrawal code 

- previous schools (2) 

- homeroom instruction 

- counsellor 

- parent /guardian information (up to 4) 

- name 

- address 

- telephone (home and business) 

- relationship 

- occupation 

- locker information 

- number 

- combination 

- student indebtedness 

- religious denomination 

- program type 

- number of credits earned 

- this school 

her schools 

- acadt ic history 

- travel information 

- method 

- distance 

- bus pass information 

- parking information 

- driver's licence 

- licence plate 

- parking space 

- medical information 

- disabilities/behaviours 

- medications 

- allergies 










74 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



WEIGHT 
(W) 



T 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 

(W X S J 
v max 7 



WT SCORE/ MAX WT SCORE 



7o 
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- date of last medical 

- physician information 

- health care number 
departure information 

- date 

- reason 

minimum of 6 user defined fields 



Instructor Information 

- instructor code 

- name 

- address 

- telephone 

- social insurance number 

- language of instruction 

- certificate number 

- courses taught 

- minimum of 6 user defined fields 
Course information 

- course code (5 character alpha-numeric) 

- description 

- pre-and co-requisites (minimum of 4) 

- must handle"and"/"or"situation 

- course type 

- language of instruction 

- course accreditation 

- credit value (2 digits) 

- pass/ fail mark 

- grade 

TOTAL Detailed Data I ten* 



15 



15 



120 



45 



19/30 



335 



450 



.744 



EVALUATION 
FACTOR 



7? 



CRITERIA ITEMS 



Reports/Inquirie r 

All reports and inquiries should be avail- 
able for all or a specified range of 
records, in various scrt orders. 

- class lists 

- homeroom lists 

- student name labels 

- student address labels 

- parent address labels 

- student I.D. cards 

- student data (alphabetical or numerical 
order) 

- parent data (alphabetical or numerical 
order) 

- instructor data (alphabetical or numer- 

al order) 

- course data 

- student phon^ li st 

- student name list 

- student grade lis*: 

- feeder school Hst 

- locker information list 

- student population oy instruction type 

- fee sheets 

The system should How production of 
user-defined reports/inquiries usi.tg 
available data. 

TOTAL Reports/Inquiries 

TOTAL SCHOOL RECORDS 



WEIGHT 
(W) 



25 



25 



90 



SCORE 
(S) 



6 

38/70 



WEIGHTED SCORE 
(W X S) 



150 



MAX WT SCORE 
< W X W 



SCORE/MAX UTT SCORE 



150 



250 



594 



900 



.66 



78 



73 
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FACTOR 




(W) 


(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

(wxs ) 

x max 


WT SCORE/MAX W7 SCORE 




SCHEDULING 














Detailed Data Items 














- Course code 
course sec l ion 














Manual scheduling (Arena Scheduling) 


7 


9 


63 








Pre-scneduling 














fVkiiirco Dp flnochc 
v/JULbc WcmJcoLo 














manual entry 


5 


9 


AS 








automated entry 


9 


9 










- allow student to specify mandatory/ 
compulsory courses, 

- preferred courses, preferred 
alternatives, etc. 

- allow student to specify preferred 
section , semester , or instructor 














Edit and validation of course requests 


7 


6 


42 








- checking of pre- And co-requisites in 
the current students' requests as well 
as history files 

- capability to override pre- and co- 
requisites 

- capability to complete pre-requlsite 
Luct.Mu^ iui suuuenLS Lruni ocrier 
District schools. 

Pre-scheduling reports 


1 

9 


8 


72 




i 




- potential conflict matrix — for all 
or a specified range of courses. 
Additional selection criteria may be 













EVALUATION 
FACTOR 



CRITERIA ITEMS 



based on thj numb?r of :equeF«_s or the 
number of sections. 

- course tally 

- students with no requests 

- student course request list 

- min/max request list 

- min/max credit list 

- verification tickets 

- arena scheduling labels 

- students missing compulsory courses 

- stu ^nts requesting specific course or 
group of courses 

Master s duU ' builder 

Capability to build a master schedule 

manually 

automatically 
Capability of handling a variety of 
Scheduling units 

c«Jl year 

- semester 

- trin. >8*:er 

- quartermester 

- 6 week un: ! t 

- any combination cf the above 

User defined timetable rotation/tum.)le 
flexible number of periods per day 
Capability to specify pxclusiv male or 
female sections 

Capability to maintain current and future 
year/serrcster master schedules 



ERLC 
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WEIGHT 
(W) 



SCORE 
(S) 



10 



19. 

5 



10 



10 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 

(W X S ) 
max' 



48 



54 



100 



100 



°0_ 
24 



WT SCORE/MAX WT SCORE 



82 



2Ek 



EVALUATION 
FACTOR 



83 
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CRITERIA ITEMS 



Scheduling Process 

User defined scheduling sequer e 

- low grades first 

- high grades first 

- A to Z 

- Z to A 

Unscheduling of no-shows /withdrawals 

Scheduling of individual student or smfll 

groups of students 

Capability to reset all students or 

partially scheduled students 

Capability to lock scheduling assignments 

for all students or a group of students 

Restart capability 

Course weighting/semester balancing 

(ensure even course Ic^c* for students) 

Blocking of courses 

Section balancing 

Class balancing (r.iales~i>malps) 

Capability to keep scheduliig opr-n kfter 

school start whjle startler rr > use the 

attendance mc/ " 

Scheduling Reports/ I nquiries 

- student ti'aetables — • grid anc list 
format 

- instructor timetables — grid ^nd list 
format 

- room timetaM — grid and list format 

- master e ' 

- student duling conflicts 

- student: partially scheduled 

- unassigned time 



"1 



WEIGHT 
(W, 



SCOR p 
(S) 



(W X 



* 1C 



10 



TO 



45 



Mi 



40 



~8 



81 

70 



MAX SCORE 



WT SCORE/MAX WT SCORE 



8 



EVALUATION 
FACTi )R 


CRITERIA TTEMS 


WEIGHT 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

(w x s TOX ) 


WT SCORE/MAX WT SCORE 




Junior riigh Scheduling Requirements 














Homeroom grouping for core subjects 














Capability of scheduling any course in 

any f^nmVW r\Sl t~ i r\r\ an/I nnmKnr r\f *- A rrw=> 
ci I y K^.\J IIIUX 1 1<1 L L \)l 1 rll 1U l lUIul/v! L U 1 L 1 TTlfcf 

periods 















TOTAL SCHEDULING 


181 


166/240 


1254 


1 

1810 


.6928 




STUDENT ATTENDANCE 














Entry of Attendance Data 














nkiuwni. ci i l l y 


J 


7 


ib 








automated entry 


9 


8 


72 








Multiple user-def inod absence types 


8 


9 


72 








ui^duiiiLy uu recoi i dLLci aance uaca at 
various intervals 


10 


9 


90 








- daily 

- tvrfce per day 

- period by period 

- subject by subject 














Attendance history 


8 


9 


7„ 








- at least ten days retail 

- curonulattve totals 














Attendance reports/inquiries 


10 


6 


60 








- student by class 

- student by subject 

- student by period 













r 8 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 


WT SCORE/MAX WT SCORE 




- homeroom attendance 

- daily summary 

- weekly summary 

- monthly summary 

- multiple absence 

- capability to produce unexcused 
absence report for the current day 
within 30 rrlnutes 

- *" v system should allow user defined 

j ipor ts/ inquiries using available data 














TOTAL ATTENDANCE 


50 


48/60 


401 


500 


.802 




STUDENT MARKS 














Entrv of nwrk*? Hafa 














manual 




8 


AO 








automated 


9 


8 


72 








Marks data 


10 


8 


80__ 








- minimum of 4 term marks plus final mark 

— Ipf |pr or nprrpnf^D'P trradp^ 














Student Zxan*^ 


6 


0 


0 








Exam timetable builder 














- automated 

- manual 
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Exam Reiorts/Inquiries 

- potential exam conflict matrix 

- exam schedules 













EVALUATION 
FACTOR 



(EASE OF 
USE 



CRITERIA ITEMS 



Reports/Inquiries 

proof list 
report cards 

- marks data 

- final r~>rk, calculated according to 
user-defined formula at^nandance data 

- cl<*ss averages 

- honour lists 

- potential failure lists 

- grad. ation list 

TOTAL STUDENT MARKS 



UTILITY rJHCTIONS 

Backup/Restore 
Security/Controls 
TOTAL UTILITY FUNCTIONS 

GRAND TOTAL, PRODUCT SCOPE AND FUNCTION 



WEIGHT 
(W) 



SCORE 
(S) 



flexibility 
modular, table driven 
help facilities 
menu driven 



CKA.ND TOTAL, EASE OF USE 



10 



381 



60 



|290/4«)| 



12 


8 


96 


8 


0 


0 


20 


8/20 


% 









40 



WEIGHTED SCORE 
(W X S) 



30/50 



60 



252 



'»80 









j 2597 




3810 



.'. WT SCORE 
(W X S^) 



400 



200 



WT SCORE/MAX Wr SCORE 



.6T 



.h8 



.68161 



1 




in 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



TECHNICAL 
CONSIDERATIONS 



SUPPORT & 
SERVICES 



WEIGHT 
(W) 



- hardware 

- system soft /are environment 

- operating system 

- utilities 

- database management/system 
internals/files 

- networking capabilities 

- user hooks 

- nodularity of the system 

GRAND TOTAL, TECHNICAL CONSIDERATIONS 



- loca^ versus where/how far 

- package support and services 

- software support, custom 
modifications 

- documentation 

- user guide, application system, 
procedural, operations guide, 
file layouts 

- training 

- applications system, operational 
(DP), availability schedule, format, 
location, prerequisites 

- implementation 

- training 

- initialisation (conversion^ ile set- 
up, output forms) 

- implementation plan 



GRAND TOTAL, SUPPORT & SERVICES 



80 



80 



70 



SCORE 
(S) 



WEIGHTED SCORE 
(V X S) 



720 



MAX WT SCORE 
< W X S max> 



WT SCORE/ MAX WT SCORE 



720 



70 



560 



560 



800 



3 



L 



700 



.8 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



PRODUCT 
QUALIFICATIONS 



VENDOR 



ERJC 



package background 
reliability 

current development status 
number of installations 
product development plans 
release concept, portability, 
verticality 



GRAND TOTAL, PRODUCT QUALIFICATIONS 



- Corporate information 

- background and history 

- financial performance 

- employee base 

- Market volatility and vendor stability 
References 
Contractual Terms 

- maintenance 





- warranty 




- ownership rights 




- discount structure/price limit 




GRAND TOTAL, VENDOR 


93 





mm 



WEIGHT 
(W) 



SCORE 
(S) 



WEIGHTED SCORE 

;w x s) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



80 



90 



70 



70 



720 



720 



560 



560 



|j»o 



700 



.9 



1)4 



Observations 



The following comments and observations are offered in support of the 
quantitative evaluation of TSS. 

(A) Product Scope and Function 

Pre-Registration: There is no distinct pre-registration function, 

rathe: , it is part of the enrolment function. 
Registration confirmation notices are not mailable • 
However, the pre-registration of a student, even 
though it requires two screens, can be done very 
expediently. 

Detailed Data Items: Demographic data is good alti ough the program does 

not handle middle names, the 15 character I.D., 
future year grade or academic history. 

Instructor information is minimal - only name, I.D. 
number and alpha-numeric instructor code are 
provided, Howevar, this information is sufficient 
for use by this system. 

Course information is adeq^te and pre- and co- 
requisites are checked a*: scheduling time which makes 
for speedy data entry. There is no limit on the 
numb' v of pre- and co-requisites. Some difficulties 
occur with courses that are offered in both full year 
and semesters, 

I eports/lnquiries : The package lacks some reports especially student 

I.D. cards and fee sheets but overall has a very 
comprehensive repertoire. In addition, we were 
unable to obtain reports for names of students in 
alphabetical or -Tumeric order. The (future release) 
Report Writer should alleviate most of these problems 
although it should be noted here that scores reflect 
the current status of the product. 

Scheduling: On the positive side, the system provides both manual 

and automated entry of course selections. The only 
minor problem is that duplicate course requests are 
not detected. 

Pre- and co-requisites are not checked in the 
academic history and th 7 can only be over-bidden at 
scheduling time. 

The Automatic Course selection feature provides a 
powerful tool for purposes such as: 

- course blocking 

- separation into male/temale only sections 

- withdrawing students from cancelled courses. 



95 (52) 



On the negative siJe, the conflict matrix is 
presented in an unsatisfactory format and there is no 
capability to automatically build a master 
timetable. The package can deal only with certain 
combinations of courses, for example no units smaller 
than quarters, not both tri- and quarter-mesters . 

Overall, Columbia provides excellent flexibility in 
terms of the timetable rotation and tumble, and the 
number of periods per day. There is a capability to 
maintain both current and future year master 
schedules and the ability to "roll over" future year 
to current year is also provided. In addition, the 
scheduling process runs extremely quickly and 
produces excellent results although it does not allow 
the user to define scheduling sequence, hovjever grade 
order can be specified (the algorithm used makes this 
feature unnecessary) . 

The scheduler provides the capability to schedule 
groups of students without affecting the timetables 
of the remainder and it does provide the ability to 
schedule any individual student. 

Scheduling Reoorts: Student timetables are available in grid format 

only. List timetables are not available in a form 
suitable for distribution to students. 



Attendance: Manual entry of attendance data requires the use of 

two screens which is awkward. Automated entry of 
attendance data is supported and is used currently at 
one of the pilot sites, although no reason codes 
(reasons for absence) can be entered from the scan 
forms . 



On the positive side, the system provides excellent 
attendance history - full year in detail plus 
cumulative totals. The attendance module is capable 
of recording absence dafs daily, twice daily and 
period by period. 

The attendance reports available are qui ce. 
comprehensive. The Report Writer package under 
development will enhance the reporting capability in 
this area. 



Student Marks: Manual and automated entry of marks data is supported 

and the system allows up to 10 term marks, letter or 
percentage grades or a mixture. Report cards ara 
produced reasonably quickly and seem adequate for 
school needs. 
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No exam timetable facilities are provided and whilp 
basic reports dre available , more detailed reports 
are required. 



Utility Functions: 



The system provides a good backup and restore 
utility. It also provide? an excellent user 
ID/password security system. 



(B) Ease of Use 



The system is very flexible allowing any timetable 
rotation and up to 4 semesters. It allows import and 
export of data to the main database using File 
Builder/Virtual Scan input. 

All functions are driven from concise menus; there is 
consistent cursor control and function key 
handling. Screen response is fast and error messages 
are generally good. 

On the negative side, there is no online help 
facility, but the documentation is excellent. 

The docunentation provides step by step descriptions 
of the functions which must be performed in order to 
run The School System. 



A multi-user version of the system is available. The 
master terminal is a PC/XT or PC/AT computer with 
dumb terminals for remaining stations. Two users can 
be supported on the IBM PC/XT and up co *> users on 
the IBM PC/AT. 

The School System is a modular, integrated system 
written entirely in "C" with extensive use ot BAT 
batch files t.) control the flow of operations. It 
has a relatively open design with virtual scan forms 
and File Builder facilities to provide for various 
enhancements to the system. 



The company is based in Vancouver with excellent 
telephone support : 

- calls are returned promptly 

- the <• .npany is always willing to help whoever calls 

- the company keeps in contact until a problem is 



(C) Technical Considerations 



(D) Sup >ort and Services 



solved. 
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In addition, the company seems capable of performing 
needed maintenance tasks. 



The design of the ^ystea allows some user 
modification and the documentation is excellent with 
4 high quality binders and few errors. There are 
particularly good descriptions of the use of the 
system. Clear Instructions are provided for the 
installation cf the system, requiring little 
knowledge of IBM PC microcomputers and PCDOS 
operating system software. The package provides the 
capability for in-house development of links with the 
IBM 4341 mainframe computers for downloading and 
uploading of data. 



The package is well tested and in production at a 
number of Canadian sites. Releases are made 
available to licensed users according to schedule and 
at regular intervals. The Company clearly has a 
well-defined and organized approach to product 
migration and enhancement,, 



The Vendor is a Canadian compaay with a strong 
background in Education Administration systems, 
starting with bureau time sharing services on 
mainframe computers. Contracts appear to be reason- 
able, although tairly rigid and the price is a little 
high. 



(E) Product Qualifications 



(F) Vendor 
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4.2,3 System Performance, Strengths and Weaknesses - TSS (Columbia) 



Key Performance Indicators 



School Test Site 


Parameter 


IBM PC/XT 


IBM PC/AT 


Jasper Place CHS 


Scheduler - Time 




3:30 hours 




Scheduler - Performance 




94% 




Scheduler - expected Perf. 


94% 


J. Percy Page CHS 


Scheduler - Time 


2:30-3:00 hours 






Scheduler - Performance 


100% 






Timetables 


5:00-6:00 hours 






Master Schedule 


0:30 hour 






Class Lists/Atten. Reg. 


5-7 min. /class 




W. P. Wagner HS 


Scheduler - Time 


- 


1 :30 hours 




Scheduler - Performance 


- 


89% 




Timetables 




11:00 hours 




Conflict Matrix 




3:45 hours 




Course lally 




0:55 hours 




Master Schedule 




1:00 hours 




Class Lists/Atten. Reg. 




9:20 hours 




Marks Registers 




9:20 hours 




Student Registers 




1:00 hour 


Victoria CHS 


Scheduler - Time 




2:10 hours 




Scheduler - Performance 




98% 




Course Tally 




0:35 hours 




Master Scher". -ie 




0:40 hou r s 




Class Lists/Atten. Reg. 




2-3 min./c 



Jasper Place CHS: 1846 students 

J. Percy Page CHS: 463 students 

W.P. Wagner HS: 975 students 

Victoria CHS: 1598 students 



(All times are in hours :minutes) 
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System Strengths ; - Meets its own specif ications 

- Well thought out data hase 

- State of the art software design 

- Generally "clean running" system 

- User definable data fields 

- Surprisingly fast in ail functions 

- Very good and consistent data entry mechanisms 

- Easy to install and learn 

- Very well documented 

- Flexible change/edit capability for student course 
requests re: 

Mass changes (by sex, grade, program) 

Semester preference 

Teacher preference 

Section preference 

Individual and global alternates 

Required course selection 

- Fast, high integrity scheduler - remains "open" 

- Excellent support, problem resolution - 1-800 hot- 
line 

- Very good approach to enhancement, planned migration 

- Open system design (facilitates future development) 

- Automated data entry and multi-user facility 

- Accomodates homeroom grouping 

- Schedules any course in any combination or number of 
periods 

- Accomodates any rotation tumble for any number of 
periods 

System Weaknesses : - Reporting limitations (format, range, common 

reports) 

- Very limited instructor data 

- Two screens required for attendance, registration 

- Some important fields absent (e.g. EPSB I.D., middle 
name) 

- No pre-requisite checking - can be honoured in 
scheduling process however 

- Credits based on semester amount - need to use 
different course codes 

- Student change transactions not captured 

- Homeroom assignment by sequential allocation or 
random - not by course section 
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4.3 Evaluation of CEMAS 



4.3.1 Testing Environment and Conditions 

The evaluation approach was one of a simulation of a real life environment. 
The test data, used in the evaluation proceso, was the Jasper Place High 
School 198 9 -84 and 1984-85 real life data. The entire student body, course 
offerings, and student course selections formed the test data base. The data 
was input into the system via the data entry functions offered by CEM\S. In 
many instances, CEMAS offe » alternative routes to get to the same point. 
Where it was feasible and/or important the alternate routes were explored to 
determine the optimum one for the future use of the system. 

The hardware unitj used in the evaluation process represented, in our opinion, 
the minimum configurations that would be required for practical use. 

All processes were simulated in their natural o~der of occurrence. The 
evaluation process took a considerably longer period of time tnan was 
originally anticipated due to circumstances beyond the control of the 
evaluation team. 

The evaluation started on an IBM PC with a 35 Mb Tallgrass hard disk and a 
(General Electric 300 cps.) GENICOM printer. Since many processes in CEMAS 
run for 6 to 10 hours (close of scheduling runs 168 hrs) and make the system 
unavailable during this time, in order to speed up the evaluation a second 
unit (IBM PC XT with 10 Mb hard disk, and a 200 cps. OKIDATA printer) was 
added. A third unit was added later and installed in Steele Heights Junior 
High School in order to determine the applicability of CEMAS to the Junior 
High School environment. 

4.3.2 Evaluation Results and Observations 

The following tables show the outcome of the quantitative evaluation of CEMAS 
against the detailed criteria* 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 

(w) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

(W X S_ a J 
max 


WT SCORE/MAX WT SCORE 


PRODUCT 
SCOPE & 
FUNCTION 


SCHOOL RECORDS 

Pre - Reds traM on /Fnrnl In^nh 














Cieate student record 


15 


7 


105 








- school student I.D. 

- last name 

- mlddl3 name 

- firct rxpane 

- birthdate 

- current grade 

- sex 

- feeder school 














Registration confirmation notice 


3 


0 


0 








Feeder school confirmation notice 


2 


0 


0 








tueal rre-Keglstratiofi/Enroll«ent 


20 


7/30 


105 


200 


.52 




Detailed Data Items 














Student information 


25 


9 


225 








- school student I.D. 

- District student l.D. 

- Alberta Education student I.D. 

- last name 

- middle name 

- first name 

- birthdate 

- current grade 

- sex 

- feeder school 
~ home address 

- telephone ntimbpr 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



WEIGHT 

(W) 



SCORE 
(S) 



WEIGH! ED SCORE 
(W X S) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 
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emergency contact 

- name 

- telephone 
entry information 

- entry date 

- registration code 

- withdrawal code 
previous schools (2) 
homeroom instruction 
counsellor 

parent /guardian information {up to 4) 

- name 

- address 

- telephone (home and business) 

- relationship 

- occupation 
locker information 

- number 

- combination 
student indebtedness 
religious denomination 
program type 

number of credits earned 

- this school 

- other schools 
academic history 
travel information 

- method 

- distance 

- bus pass information 
parking information 

- driver e s licence 

- licence plate 

- parking space 
medical infomtion 

- disabilities/behaviours 

- medications 

- allergies 



05 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SftRE 

(w x sj 


MAX WT SCORE 

(w x Smax ) 


WT SCORE/MAX WT SCORE 

i 
I 




- date of last medical 

- physician information 

- health care number 

- departure information 

- date 

- reason 

- minimum of 6 user defined fields 










i 




Instructor Information 


5 


2 


10 




■ 




- instructor code 

- name 

- address 

- telephone 

- social insurance number 

- language of instruction 

- certificate number 

- courses taught 

- minimum 1 6 user defined fields 










; 




Course information 


15 


1 


15 








- course code (5 haracter alpha-numeric) 

- description 

- prj-and co-requisites (minimum of A) 

- must handle^'andVor^situation 

- course type 

- language of instruction 

- course accreditation 

- credit value (2 digits) 

- pass/fail mark 

- grade 














TOTAL Detailed Da* a It era 


45 


12/30 


250 


450 


.55 


.106 
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CRITERIA ITEMS 



Reports/Inquiries 

All reports and inquiries should be avail- 
able for all a specified range of 
records, in various sort orders. 

- class lists 

- homeroom lists 

- student name labels 

- student address labels 

- parent address labels 

- student I.D. cards 

- student data (alphabetical or numerical 
order) 

- parent data (alphabetical or numerical 
otder) 

- instructor data (alphabetical or numer- 
ical order) 

- course data 

- student p list 

- student name list 

- student grade list 

- feeder school list 

- locker information list 

- student population by instruction type 

- fe« sheets 

Th2 system should allow production of 
user-defined reports/inquiries sing 
available data. 

TOTAL Reports/Inquiries 

TOTAL SQtOOL RECORDS 



WEIGHT 
(W) 



25 



25 



90 



SCORE 
(S) 



20/70 



WEIGHTED SCORE 
(W X S) 



25 



MAX WT SCORE 



(W 



X S ) 
max' 



WT SCORE/MAX Wr SCORE 



25 



380 



250 



900 



.42 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

(W X S ) 
v max' 


WT SCORE /MAX WT SCORE 




SCHEDULING 














Detailed Data Items 














- Course code 

- Course section 














Manual scheduling (Arena Scheduling) 


7 


8 


56 








Pre-scheduling 














Course Requests 














manual entry 


5 


4 


20 








automated entry 


9 


0 


0 








- allow student to specify mandatory/ 
compulsory courses, 

- preferred courses, preferred 
alternatives, etc. 

- allow student to specify preferred 
section, semester, or instructor 














Edit and validation of course requests 


7 


3 


21 








- checking of pre- and co-requisites in 
the current students' requests as well 
as history files 

- capability to override pre- and co- 
requisites 

- capability to complete pre-requisite 
checking for students from other 
District schools. 

Pre-scheduling reports 


9 


2 


18 






.110 


- potential conflict matrix — for all 
or a specified range of courses. 
Additional selection criteria may be 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 

< w x w 


WT SCORE/MAX WT SCORE 




based on the number of requests or the 
number of sectiot ~. 

- course tally 

- students with no requests 

- student course request list 

- min/max request list 

- min/roix credit list 

- verification tickets 

- arena scheduling labels 

- students missing compulsory courses 

- students requesting specific course or 
group of courses 














Master schedule builder 














Capability to build a master schedule 
manually 


6 


8 


48 








automatically 


9 


1 


9 








Capability of handling a variety of 
Scheduling units 


9 


0 


0 








- full year 

- semester 

- trimester 

- quartermester 

- 6 week unit 

- ^ny combination of the above 














User defined timetable rotation/ tumble 


10 


0 


0 








Flexible number of periods per day 


10 


4 


40 








Capability to specify exclusive male or 
female sections 


5 


6 


30 
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Capability to maintain current and future 
year/semester master schedules 


8 


1 


8 
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EVALUATION 
FACTOR 
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CRITERIA ITEMS 



Scheduling Process 

User defined scheduling sequence 

- low grades first 

- high grades first 

- A to Z 

- Z to A 

Unscheduling of no-shows/wirhdrawals 

Scheduling of individual student or small 

groups of students 

Capability to reset all students or 

partially scheduled students 

Capability to lock scheduling assignments 

for all students or a group of students 

Restart capability 

Course weighting/semester balancing 

(ensure even course load for students) 

Blocking of courses 

Section balancing 

Class balancing (males-females) 

Capability to keep scheduling open after 

school start while starting to use the 

attendance module 

Scheduling Reports/Inquiries 

- student timetables — grid and list 
format 

- instructor timetables — grid and list 
format 

~ room timetables — grid and list format 

- master schedule 

- student scheduling conflicts 

- students partially scheduled 

- unas signed time 



WEIGHT 
(W) 



10 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



24 



30 



42 



40 



56 



32 



MAX WT SCORE 

(W X S ) 
v max' 



WT SCORE/MAX WT SCORE 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



Juuxui Kifeii Sch eduling Kequirements 

Homeroom grouping for core subjects 
Capability of scheduling any course in 
any combination and number of time 
periods 

TOTAL SCHEDULING 



J 



9 

ERIC 
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STUDENT ATTENDANCE 

Entry of Attendance Data 



manual entry 
automated entry 

Multiple user-defined absence types 

Capability to record attendance data at 
various intervals 

- daily 

- twice per day 

- period by period 

- subject by subject 

Attendance history 

- at least ten days detail 

- cummulative totals 

Attendance reports/inquiries 

- student by class 

- student by subject 

- student by period 



WEIGH"* 
(W) 


scori; 

(S) 










181 


73/240 


5 


6 


9 


C 


8 


0 


10 


6 


8 


3 


10 


2 







WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



490 



30 



60 



1810 



24 



20 



.2707 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(V) 


SCORE 
CS) 


WEIGHTED SCORE 
(V X S) 


MAX WT SCORE 

< w x W 


WT scorE/MAX r srnR*? 




~ homeroom attendance 

- daily summary 

- weekly summary 

- monthly summn r y 

- multiple absence 

- capability to produce unex r used 
absence report for the current day 
within 30 minutes 

- the system should allow user defined 
reports/inquiries using available data 














TOTAL ATTENDANCE 


50 


17/60 


134 


yo 


.268 




STUDENT MARKS 














Entry of marks data 














manual 


5 


4 


20 








automated 


9 


! o 


o 








Narks dtta 


10 


8 


80 








- .minimum A term marks plus final mark 

- letter or percentage grades 














Student Exans 






18 








Exam timetable builder 














- automated 

- manual 














Exam Reports/Inquiries 














- potential exam conflict matrix 

- exam schedules 












118 
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EVALUATION 
FACTOR 



EASL OF 
USE 



120 
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CRITERIA ITEMS 



Reports/ Inquiries 

proof list 
report cards 

- marks data 

- final mark, calculated according to 
user-defined formula attenandance data 

- class averages 

- honour lists 

- potential failure lists 

- graduacion list 

TOTAL STUDENT MARKS 



UTILITY FUNCTIONS 
Backup/Restore 
Security/Controls 
TOTAL UTILITY FUNCTIONS 



WEIGHT 
(W) 



10 



40 



12 



20 



GRAND TOTAL, PRODUCT SCOPE AND FUNCTION 



- flexibility 

- modular, table driven 

- help facilities 

- menu driven 



GRAND TOTAL, EASE OF USE 



381 



60 



60 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



15/50 



10/20 



135/440 



118 



48 



48 



% 



1218 



180 



180 



MAX WT SCORE 
< W X S max ) 



400 



200 



3810 



600 



WT SCORE/MAX WT SCORE 



.29 



.3197 
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EV ALUATION 


CKITFRIA ITO1S 


WEIGH! 1 


SCORE 


FA^R 




(W) 


(S) 


XtUtNltAL 


- hardware 


80 


2 


OONSIDERATIONS 


- system software environment 








- operating system 








- utilities 








- database management/system 








internals/ files 








- networking cabpabilities 








- user hooks 








- modularity of the system 








uKAND TulAL, TECHNICAL CONSIDERATIONS 


80 


n 

i 


oUrrUKl ft 


— local versus where /how far 


70 


0 


SEEVTCES 


- package support and services 








- software support, custom 








modifications 








- documentation 








- user guide, application system, 








procedural, operations guide, 








file layouts 








- training 








- applications system, operational 








(DP;, availability schedule, format, 








location, prerequisites 








- implementation 








— training 








- initialization (conversion, file set- 








up, output forms) 








- implementation plan 






122 


C -AND TOTAL, SUPPORT & SERVICES 


70 

I 


1 
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WEIGHTED SCORE 
(W X S) 



160 



160 



MAX WT SCORE 
< W X S max> 



WT 



800 



700 



SCORE/MAX WT SCORE 



.2 



23 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



WEIGHT 
(W) 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



PRODUCT 
QUALIFICATIONS 



VENDOR 



9 

ERLC 



80 



package background 
reliability 

current development status 
number of installations 
product development plans 
release concept, portability, 
verticality 



GRAND TOTAL, PRODUCT QUALIFICATIONS 



80 



800 



70 



- Corporate information 

- background and history 

- financial performance 

- employee base 

- Market volatility and vendor stability 

- References 

- Contractual Terms 

- maintenance 

- warranty 

- ownership rights 

- discount structure/price limit 



GRAND TOTAL, VENDOR 



70 



70 



[ 



70 



700 
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Observations 

The following comments are offered in support of the quantitative evaluation 
of CEMAS. 

(A) Product Scope and Function 

Pre-Registration: The facility is good but there is no method of 

producing the appropriate reports. There is an 
excellent range of student demographic data but no 
user defined fields. Particularly important 
fields missing include: middle name, 15 character 
EPSB I.D. 

Detailed Data Items: Most important fields are present; again, the lack 

of user-defined fields causes problems. 

Instructor Information: The instructor information is good but is 

restricted to numeric teacher codes and again no 
user-defined fields. 

Course Information: The data in this area is inadequate, for example: 

- Credit range is 0.000 to 9.999 which is 
insufficient for large courses which can have up 
to 30 or more credits. 

- It does not allow co-requisite<= • 

- Only 2 pre-requi sites per course are allowed. 
Reports/inquiries: There were numerous problems with reports: 

- They do not always work for a range of values. 

- They are very slow in most cases. 

- Grid timetables do not recognize school 
timetable "tumbles" , 

Many reports do not exist , in the system and there 
is no mechanism available for their derivation; 
for example there is no Report Writer program. 
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Pre-Scheduling: The manual entry of course requests is slow and 

there is no facility for the automated entry of 
course requests • In add it ion , it is impossible to 
create academic history to test the ability to 
check pre-requisites from previous history; the 
package cannot check pre- and co-requisites of 
courses in the given year, i.e. it cannot force 
Chemistry 10 to be scheduled before Chemistry 20 
if both are requested. 

A potential conflict matrix can be produced but 
only for all courses; one cannot specify a range 
of courses. 

Master Scheduler Builder: 

An automatic schedule builder is provided but it 
does not work. The system, in thi? function area, 
can only handle full year and semestered courses, 
not quarter-mesters or tri-mesters. A mechanism 
exists for providing very simple timetable 
rotations but even this is not reflected in the 
student grid timetables, marks or attendance 
lists. 

A mechanism also exists for maintaining current 
and future year master schedules but this does not 
work correctly. 

Scheduling Process: The scheduling sequence cannot be directly 

specified, this can be achieved indirectly with 
"patching" tricks. The process itself is slow and 
cannot be restarted if aborted. Multiple passes 
are needed. The process does not work correctly; 
in some cases it puts students in two classes at 
the same time and ignores some sections of courses 
resulting n unbalanced classes. 

Course blocking facilities are provided but do not 
work properly. Also, one caanot keep attendance 
until scheduling has been closed . Scheduling 
"close" ties up the system for an unacceptably 
long period of time (1 week at Jasper Place High 
School). 

Scheduling Reports/Inquiries: 

Scheduling reports are very slow and many do not 
work. There are no user definable reports. 
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Attendance: As described above, attendance cannot be kept 

until scheduling is fully closed* There is no 
mechanism for automated attendance data entry and 
manual data entry is slow. In addition, there are 
no user definable attendance type codes. 

It was impossible to test the attendance history 
function thoroughly since the attendance "aging" 
function does not work properly. 

Attendance reports are v^ry slow and some optirns 
do not work. There are no user-definable reports. 

Student Marks: There is :> function for automatically entering 

marks data; manual entry is slow. An exam 
timetable builder is provided but does not work. 

The report cards function does not work due to the 
absence of format specifications • 

Utility Functions: Backup and Restore functions are not provided; the 

standard utilities provided under PC DOS are 
adequate for programmer use. 

The security syster is adequate although it does 
not appear to function as stated in the 
documentation. 

The need to re-start the application package after 
using utility functions i . annoying and would be 
particularly difficult for a non technical user. 

(B) Ease of Use: 

CEMAS is not a flexible system - it can only 
handle certain types of timetable rotations and it 
only allows 2 semesters. Although menu-driven, 
the function keys and cursor controls are not 
consistent from screen to screen. 

Tne online help facility simply lists sections of 
the operating manual and error messages are often 
cryptic or inappropriate with no explanations in 
the documentation. Screen response is generally 
slow. 

The overall ease of use is negatively impacted by 
poor system performance and poor functionality, 
and the need to avoid functions that do not work. 
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(C) Technical Considerations: 



The system runs on an IBM PC/XT under PC DOS 2.00 
or higher operating system version. CEMAS was 
tested In a single use*: environment only. It is 
unable to extract data from external files as no 
information was given on file layout or content. 

Overall, the CEMAS package is a closed system with 
a non -modular design. Tnis causes difficulties 
for the vendor in making modifications; it is 
impossible for the user to do this. 



The company is based in Toronto and is fairly 
small, leaving doubts as to their ability to make 
custom modifications. Telephone support is 
poor. 

For example: 

- Calls were not returned 

- We were frequently able to reach only the 
answering service 

- There was a tendency to blame problems on user 
error rather than to admit the possibility of 
problems with the software 

- The company was sometimes reluctant to talk to 
the Analyst /Programmer who cal^d, they 
preferred to talk to the Project Leader only. 

The design of thG system does not lend itself to 
user modification and there is no ability to set 
up system data except by keypunch. 

The documentation is poor in appearance and 
substance. For example: 

- There are many typographical errors 

- Thera is no index 

- Thrre are no instructions given for installation 
of the system 

~ Descriptions of functions and their use are 
inconplete and poorly explained 

- On-line help consisted of displaying the 
appropria te sect ion of the printed documentat ioi 

- There vras no explanation of error messages 



(D) Support and Services: 
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(E) Product Qualifications: 



During the course of the CEMAS investigation, many 
problems were encountered with both the product 
and with vendoi support, which extended the 
evaluation process considerably. As a 
consequence, one very important conclusion which 
was drawn by the team is that CEMAS was a product 
which was still under active development. The 
frequency and nature of product updates has 
clearly supported this conclusion. Updates 
received during the course of the evaluation would 
be best described as fixes rather than product 
enhancements. Product updates were occasionally 
found to corrupt things which had previously 
worked. Up to and including the final days of 
practical testing of the product, it is the 
opinion of the evaluation team that CEMAS was not 
a mature or stable product. 

We are unable to say how many production sites are 
currently using CEMAS. 

F) Vendor The vendor, which is also the developer - 

COMPUTERLJB, is based in Toronto. To date there 
is no known local support for CEMAS. Systems 
documentation is poor but was improving. The 
Distributed Systems Team was in frequent and close 
communication with the developers throughout the 
evaluation of CEMAS. The frequent problems which 
were encountered were communicated to Computerlib 
with expedience. Response to problem reporting 
was mixed at best, and problem resolution was less 
than acceptable. Lack of effective, local support 
for CEMAS should be considered a significant 
inhibitory factor to potential users and this 
factor becomes even more critical where District 
level support is unavailable. 



9 

ERLC 



J30 

(75) 



4.3.3 System Performance^ Strengths and Weaknesses - CEMAS (Coraputerlib) 
Ke^y Performance Indicators 

School Test Site 

Jasper Place 



Parameter 


IBM PC/XT 


Timetables 


doesn't work 


Conflict Matrix 


6:00 hours 


Course Tally 


3:00 hours 


Master S:hedule Print 


2:00 hours 


f 1 ! ace T i cfc 


1 — X n /r1 ace 
i -J llli.ll/ l_J.aoo 


Attendance 


1-3 min/class 


Marks Registers 


1-3 min/class 


Student Registers 


1-3 min/class 


Course Requests 


27.0 hours 


Scheduler - Time 


19:30 hours 


Scheduler - Performance* 


84% 


Scheduler - Expected 
Performance 


85% 



Jasper Place CHS: 1846 students 
(All times are In hours: minutes) 
N.B. Subsequent runs corrupted previously achieved results 
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System Strengths ; - Comprehensive database with good data elements 

- Easy to use creens 

- Well integrated system; modules all fit together 

System Weaknesses : - System still under conversion and/or development 

- Unavailability/non-existence of system documentation 

- System does not use Miudle Name - link to mainframe not 
possible 

- The scheduler may be limited to 8 periods per day 

- Course credit format is N.NN; should be NN 

- Unable to generate ad-hoc reports 

- Benchmark tests (particularly scheduling) not complete' 

- Some functions not working (e.g. student request list) 

- Report production tiae long - cannot be effectively 
suspended 

- Many system functions are very slow 

- Master schedule builder does not accomodate semester/ non- 
semester mix 

- Course translation not available 

- Doesn f t handle quarter-semester courses 

- "fatal" errors occur without warning 

- Unstable paging condition during report production 

- Instructor code presently numeric - needs at least to be 
alphanumeric 

- References on installed IBM systems not available 

- Hard coding of year into system 

- Must enter area code with eveiy telephone number 
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5 .0 COMPARATIVE EVALUATION OF SIMS; SENIOR HIGH SCHOOL PERSPECTIVE 



5 . 1 Comparison Summary and Review of SIMS Evaluation Da> . a 

Tue following tables show the quantitative evaluation data for the three 
microcomputer based information management systems which were 
evaluated. Two mini-computer products were also tested in the same 
environment and will be the subject of another report. The data is 
displayed on the Detailed Scoring Comparison Form which was referred to 
previously. This form parallels the Detailed Evaluation Criteria 
Forms. The Comparison Summary and Review forms differs from the Detailed 
Criteria Forms in that all (non-scorable) context related criteria are 
omitted and only the weighting factor, raw and weighted scores from the 
evaluation are displayed. Various levels of totals are shown on the form 
the major purpose of which is to facilitate the quick and objective 
comparison of system performance. 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


TSS 

WEIGHTED 
SCORE 

(w X s) 


SCORE 
(S) 


SIRS 

WEIGHTED 

crnop 

oLURE 

(W X S) 


GEMAS 

SCORE WEIGHTED 
SCORE 

(s; (w x s) 




PRODUCT 

dwiC m 

FUNCTION 


SCHOOL RECORDS 

Pre-Re^ist rat ion/Enrollment 
















a 

<x> 

rr 

SJ 

M« 

y-> 

(U 




Create student record 


15 


6 


90 


8 


120 


7 


105 


CL 
f O 

n 




registration confirmation notl:e 
Feeder school confirmation notice 


3 


5 


15 


2 


6 


0 


0 


0 

•-t 




2 


2 


4 


0 


0 


0 


0 


CP 




TOTAL Pre-Regl8tratlon/Enroll»ent 


20 


13/30 


109 


10/30 


126 


7/30 


105 


o 

0 
3 




Detailed Data Icems 
















G> 

•i 

M« 
0) 
0 




Student Information 


25 


8 


200 


8 


200 


9 


225 


D 
1 

cn 

0) 
D 




Instructor Information 


5 


J 


15 


7 


35 


2 


10 




Course information 


15 


8 


120 


8 


120 


1 


15 


o 
►1 

ac 




TOTAL Detailed Data Iteas 


45 


19/30 


33 e 


23/30 


355 


12/30 


250 


H- 

•■yo 

=r 

CO 

o 




Reports/Inquiries 


25 


6 


150 


6 


150 


1 


25 


0 
0 
>— 1 




TOTAL Reports/Inquiries 


25 


6/10 


i50 


6/10 


ISO 


1/10 


25 






TOTAL SCBXJL RECORDS 


90 


38/70 


594 


39/70 


631 


20/70 


380 


•i 

w 

ID 




SCHEDULING 
















n 
rr 

< 




Manual scheduling (Arena Scheduling) 


7 


9 


63 


9 


63 


8 


56 










1 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


SCORE 
(S) 


SIRS 
WEIGHTED 
SCORE 

(w X s> 


CEHAS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 




Pre-scheduling 


















M nil V* O A V A fl t 1 A 0 ^ A 

vOUlSc I\P(jU6S L 8 


















manual entry 


5 


9 


45 


7 


35 


4 


20 




auLornaLeo entry 


o 


9 


81 


*» 


JO 


0 


0 




Edit and validation of com se requests 


7 


6 


42 


9 


63 


3 


21 




T5 V* A v ^ A V% A /I t 1 1 4 A #T V* A A A V* A 

rre scneauiing reports 


Q 


8 


72 


a 


81 


2 


18 






30 


32/40 


240 


29/40 


215 


9/40 


59 




Master Schedule Builder 












v^apaoiiiLy to duuq a roaster scneouier 
manually 


6 


8 


48 


7 


42 


8 


48 




A t 1 A fft A ^ j A All 

automat icaiiy 


Q 


0 


0 


0 


0 


1 


9 




Capability of handling a variety of 
scheduling units 


Q 


6 


54 


8 


72 


0 


0 




User defined timetable rotation/tumble 


10 


10 


100 


8 


80 


0 


0 




r icAiuie numoer or penoas per aay 


in 


10 


100 


8 


80 


4 


An 




Capability to specify exclusive male or 

C Am f A A A A 4 AM A 

remaie sections 


J 


4 


20 


8 


40 


6 


30 




Capability to maintain current and future 
year/ semester master schedules 


o 
o 


3 


24 


10 


80 


1 


8 




TOTAL Master Schedule Builder 


57 


41/70 


346 


49/70 


394 


20/70 


135 




jcneauiinR rrocess 


















user aerineo scneauiing sequence 


o 


5 


30 


7 


42 


4 


24 J ! 


Unschedul inff of no— shows/wl thdrAualfi 


5 


9 


45 


9 


45 


6 


30 


1 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 

(w) 


TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


1 

SIRS 

SCORE WEIGHTED 
SCORE 
(S) (V X s) 


CEMAS 

SCORE WEICHTED 

(S) (W X S) 




Scheduling of individual student or small 

groups of students 

Capability to reset all students or 

partially scheduled students 

Capability to lock scheduling assignments 

for all students or a group of students 

Restart capability 

Course weighting/semester balancing (ensure 
even course load for students) 
Blocking of courses 
Section balancing 
Class balancing (males-females) 
Capability to keep scheduling open after 
school start while starting to use the 
attendance module 


6 


8 


_48 


n 


n 

\J 


/ 


42 




8 


10 


80 




U 


5 


40 




o 
o 


5 


AO 


0 


0 


7 


56 




8 


0 




U 


U 


1 


8 




8 


7 


56 


7 




4 


32 




7 


9 


63 


8 


56 


0 


0 




o 


8 


64 


q 


7 9 
/ Z 


0 


0 




4 


7 


28 


8 


32 


2 


8 




9 


9 


81 


0 


0 


0 


0 




TOTAL Scheduling Process 


77 


77/110 


535 


48/110 


303 


36/110 


240 




Scheduling Reports/Inquiries 


10 


7 


70 


9 


90 


0 


0 




Junior Hi*h Scheduling Reauirements 


















Homeroom grouping for core subjects 
Capability of scheduling anv murnm in 
combination and number of time periods 




























TOTAL SCHEDULING 


181 


166/240 


1254 


144/240 


1065 


73/240 


490 




STUDENT ATTENDANCE 


















Entry of Attendance Data 


















manual entry 
automated entry 


c 

J 


7 


35 


9 


45 


6 


30 




9 


8 


72 


0 


0 


0 


0 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 

(w) 


TSS 

SCORE WEIG rED 
(S) (W X S) 


SCORE 
(S) 


SIRS 
WEICHTED 

(w X s) 


CEMAS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 




Multiple user-defined absence types 


8 


9 


72 


8 


64 


0 


0 




Capability to record attendance data at 
various intervals 


10 


9 


90 


q 


90 


6 


60 




Attendance history 


8 


9 


72 


7 

f 


JU 


3 


24 




Attendance reports/ inquiries 


10 


6 


60 


7 

/ 


70 


2 


20 




TOTAL ATTENDANCE 


50 


48/60 


401 


AO/GO 


325 


17/60 






STUDENT MARKS 


















Entry of marks data 


















manual 


5 


8 


40 


8 


40 


4 






automated 


9 


8 


72 






0 


0 




Marks data 


10 


8 


80 




80 


8 


80 




Student Ixams 


6 


0 


0 


0 


0 


3 


18 




Exam timetable builder 
Exam Reports/Inquiries 


















Report s/lnquiries 


10 


6 


60 


D 
O 




0 


0 




TOTAL STUDENT MARKS 


40 


30/50 


252 


24/50 


200 


15/50 


118 


















J 4l 



pV ALU AT I ON 
FACTOR 



oo 



EASE OF 

USE 



TECHNICAL 
CONSIDERATION 



CRITERIA ITEMS 



UTILITY FUNCTIONS 

Backup/Restore 
Security/Controls 
TOTAL UTILITY FUNCTIONS 

GRAND TOTAL, PRODUCT SCOPE AND FUNCTION 



WEIGHT 
(W) 



12 



GRAND TOTAL, EASE OF USE 



GRAND TOTAL, TECHNICAL CONSIDERATIONS 



SUPPORT 
SERVICES 



GRAND TOTAL, SUPPORT fc SERVICES 



TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 



8 96 



8 


0 0 


20 


8/20 96 






381 


290/440 


- 

2597 


60 


8 480 






60 


8/10 


480 


80 


9 720 






80 


9/10 


720 J 


70 


8 560 






70 


8/10 


560 | 



SIRS 

SCORE WEIGHTED 
SCORZ 
(S) (W X S) 



8 96 



2 


lb 


10/20 


112 




257/440 


| 2333 | 



360 



6/10 



360 



480 



6/10 480 



490 



7/10 



490 



CEMAS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 



48 



48 



10/20 96 



135/440 



1218 



180 



3/i0j|~i80 



160 



2/10 



160 



0/10 



ERJC ttZ 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



PRODUCT 
QUALIFICATIONS 



CD 



GRAND TOTAL , PRODUCT QUALIFICATIONS 



VENDOR 



GRAND TOTAL, VENDOR 



4 4 



9 
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TSS 

SCORE WEIGHTED 
SCORE 
(S) (V X S) 



9/10 



720 



720 



560 



8/10 



560 



SIRS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 



6/10 



7/10 



480 



480 



490 



490 



CEMAS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 



0/10 



1/10 



70 



70 



11 J 



5.2 Relative Suitability of SIMS to the Senior High Schools 



The foregoing results, can now be used to determine the relative 
suitability of a particular product to a particular user's needs. 

The following describes a method of determining this suitability relative 
to the six major evaluation factors. 

Before determining the overall suitability of a system to the needs of 
the user, however, the user must first define the relative emphasis that 
he wishes to place on the major evaluation factors. 

The following table shows the emphasis which the evaluation team believes 
is an appropriate emphasis to place on the major evaluation factors. The 
emphases are expressed as percentages and total to 100. While it can be 
clearly seen that product scope and function is the single most important 
evaluation factor, this importance is outweighed by the collective 
emphasis on the other five factors. 



EVALUATION FACTOR 


EMPHACIS (%) 


PRODUCT SCOPE AND FUNCTION 


45 


EASE OF USE (OF PRODUCT) 


10 


TECHNICAL CONSIDERATIONS 


10 


SUPPORT AND SERVICES 


15 


PRODUCT QUALIFICATIONS 


10 


VENDOR 


10 



Relative suitability can be defined as a function of weighted score and 
relative emphasis in the following way. 



Relative Suitability - (Z Emphasis) x ( weighted score) 

(max. possible weighted score) 



The ratios of weighted score to maximum possible weighted score for the 
products evaluated are shown on the Detailed Evaluation Criteria Forms 
(sections 4.1.2, 4.2.2, and 4.3.2). 

Applying the above formula to the evaluation data at hand gives the following 
result. 
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FVAT FTATTON CAPTOR 


(%) 


RELATIVE PRODUCT SUITABILITY 


TSS 


SIRS 


CEMAS 


PRODUCT SCOPE AND FUNCTION 
EASE OF USE 

TECHNICAL CONSIDERATIONS 
SUPPORT AND SERVICES 
PRODUCT QUALIFICATIONS 
VFNDOR 


45 


30 


27 


14 


10 


8 


6 


3 


10 


9 


6 


2 


15 


12 


10 


0 


10 


9 


6 


0 


1 f) 


8 


7 


1 










TOTALS 


100 


V" 


62 


20 



By using this process, entries in the columns identified by product names 
will be numbers less than or equal to the percent emphasis number. These 
numbers can be considered as scores out of the assigned percent emphasis 
numbers. Vertical totals of suitability for each product will be numbers 
less than or equal to 100 which can easily be compared across products. 

The above table shows, for example, that CEMAS is considered to be very 
unsuitable to the needs as defined in the support and services area 
while, by contrast, Columbia's The School System scored 12 of a maximum 
possible 15 points for the same evaluation factor. 

The suitabilities calculated according to the method described should be 
viewed as relative measures of the extent to which a product meets a 
particular user's needs. This suitability will vary according to the 
completeness of the criteria, user defined weighting factors, percent 
emphasis and, very obviously, on the scores assigned by the product 
evaluator. Within this context, therefore, it is very important to note 
that the evaluation process which has been developed and applied in this 
way is extremely flexible allowing the user complete discretion to decide 
which criteria will be used, the weighting factors and the relative 
emphasis. In short, all that a user of this process needs to depend on 
is the actual raw scores which were assigned as a result of the hands-on 
testing work. 



ERLC 



147 

(86) 



To illutrate the flexibility of the process, two more examples of 
product suitability have been determined and ,*re shown below. The reader 
will see that tne percent euiphasis distribution 1 ~s been changed (vMle 
dtill totalling 100) in each case. In these examp A es, the individual 
criteria weighting f acr s were not changed (though they could have beeu) 
ari thus the same ratios of weighted score to maximum weigh, d scores 
were applied. 

SIMULATION 1 SENIOR HIGH SCH00I PERSPECTIVE 



EVALUATION FACTO* 


EMPHASIS 
(%) 


RELATIVE PRODUCT SUITABILITY 


j TSS 


SIRS 


CEMAS 


p R0DUCT SCOPE AND FUNCTION 
EASE OF USE 

TECHNICAL CONSIDERATIONS 
SUPPORT AND SERVICES 
PRODIK* QUALIFICATIONS 
VENDOR 


55 


37 


33 


17 


20 


16 


12 


6 


5 


'4 


3 


1 


:o 


8 


7 


0 


5 




3 


0 




4 


3 


0 












TOTALS 


100 


I 

7J 1 61 
1 


24 


SIMULATION SENIOR HIGH SCHOOL PERSPECTIVE 




EVALUATION FACTOR 


EMPHASIS 
(%) 


RELATIVE PRODUCT SUITABILITY 


TSS 


SIRS 


CEMAS 


PRODUCT SCOPE AND FUNCTION 
EASE OF USE 

TECHNICAL CONSIDERATIONS 
SUPPORT /.%'D SERVICES 
PRODUCT QUALIFICATIONS 
VENDOR 


50 


34 


30 


15 


20 


16 


12 


6 


10 


9 


6 


2 










20 


18 


12 


0 


















TOTALS 


1 

100 | 


77 


60 


23 



(87) J 48 



As previously stated, the approach used to define relative suitability is 
very flexible and may be employed to meet the needs of a particular 
user. Appendix 6 shows a further simulation (use of the same evaluation 
data) in which not all of the evaluation criteria were used and in which 
the actual criteria weighting factors we^e changed to reflect a particjlar 
user perspective. 
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6.0 



PRODUCT EVALUATIONS - JUNIOR HIGH SCHOOL PERSPECTIVE 



Two of the three microcomputer - based systems - SIRS and The School 
System were tested in a Junio High School in addition to the above tests 
in Senior High Schools. 

The Detailed Evaluation Criteria Forms show two specific requirements in 
relation to the scheduling function which were considered to be of 
special relevance to junior high school environments, not?bly: 

- Homeroom grouping fo^ core subjects 

- Capability of scheduling any course in any combination and number of 
time periods 

It war obviously impossible to retest these feacures and other junior 
b'gh specific features (such as morning/af ternoon attendance) with an 
existing senior high school database. For this reason the two above 
systems were tested independently in a junior nigh school. 

6.1 Evaluation of SIRS 

The MIG SIRS package was evaluated at Steele Heights Junior High School 
to determine its suirability in a junior high setting. Prior to starting 
the evaluation a discussion with MIG indicated that upgrades to the 
software would be needed in the "core subject" grouping area. These 
upgrades were mr.de about half-way through the evaluation but did not 
significantly improve th<5 overall result. 

The evaluation spanned a period of one and a half months during which 
time a number of scheduling simulations were made with improved results 
on each occasion. Current schedules and student demographic data were 
used with an expectation of achieving at least 95% fully scheduled 
student course requests • 
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( 19) 



6.1.1 Testing Environment aad Conditions 



Steele Heights Junior High school lias 646 studeiits enrolled in grades 7, 
8 and 9. It uses a strict 4 day, o period rotation schedule and operates 
attendance at the half-day reporting period level. The classes and 
subjects offered are very typical of other district junior high schools 
with a small number of ESL and vocational courses and a high correspon- 
dence to the Alberta Education course Jistings. Steele Heights uses 
\omerooms of approximately 25 students each and has, in common with most 
other Junior High schools in the Llstrict, a large number of "core" 
periods for each student, that is, mandatory courses. Grade 7 and 8 
students must take Physical Education, Computer Studies, Language Arts, 
Mathematics, Science and Social Studies. Grade 9 students do not have to 
r^ke Computer Studies, but the other five courses apply. 

The SIRS system was copied onto an IBM XT computer. The computer has * 
10 Megabyte disc drive, a 360 kilobyte flexible disc drive (used mainly 
for loading data to and c rom other sources and backing up the SIRS 
database) an amber high-resolution monitor, 512 kilobytes of RAM (Random 
Access Memory) and an OKI DATA Microline 84 printer operating at about 160 
characters per second. The system supports a single user with no option 
to upgrade to multiple users. 

Data entry was entirely by keyboard for this test, although the senior 
high school test employed da..a loading from mainframe files (which were 
initially converted to text format). 

The testing timescale was one and a half months which allowed time for 
the setting up of all demographic data, master schedule, course requests 
and school data (such as rooms, teachers, programs etc.). It also 
allowed time for 2 scheduling simulations for all students, a number of 
smaller simulation tests and limited testing of the marks and attendance 
software • 



6.1.2 E valuation Results and Observations 

The following tables show the outcome of the quantitative evaluation of 
SIRS against the Detailed Evaluation Criteria. 
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ETALUAVTON 
FACTOR 



CRITERIA ITEMS 



rooocr 

SCOPE & 
FUNCTION 



SCHOOL RECORDS 

Pre-Registration/Enrollment 
Cre .e student record 

- school student I.L. 

- last name 

- middle -lame 

- first name 

- birthdate 

- current grade 

- sex 

- feeder school 

- home address 

Registration confirmation notice 
Feeder school confirmation notice 

TOTAL Pre-Reglstratioo/Enrolljient 

Detailed Data Items 

Student information 

- school student I.D. 

- District student I.D. 

- Alberta Education student I.D. 

- last name 

- middle name 



ERLC 





- birthdate 




- current grade 




- sex 




- feeder school 




~ home address 




- telephone number 


152 





WEIGHT 
(V) 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 
< W X W 



WT SCORE/MAX WT SCORE 



15 



120 



20 



8/30 



120 



200 



25 



100 



1 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE 
< W X W 


WT SCORE/MAX WT SCORE 


15 i 


- emergency contact 

- name 

- telephone 

- entry Information 

- entry date 

- registration code 

- withdrawal code 

- previous schools (2) 

- homeroom instruction 

- counsellor 

- parent /guard! an infot^tion (up to 4) 

- name 

- address 

- telephone (home and business) 

- relationship 

- occupation 

- locker information 

- number 

- combination 

- student indebtedness 

- religious denomination 

- program type 

- number of credits earned 

- this school 

- other schools 

- academic history 

- travel information 

- method 

- distance 

- bus pass information 

- parking information 

- driver's licence 

- licence p] ite 

- parking "pace 

- medical information 

- A\ eahi 1 ^ t" i vl our r 

U1DOU111L1C9/ i^Z lid V XVJLli. O 

- medications 

- allergies 










15o 



EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 

(W) 


SOORE 
(S) 


WEIGHTED SOORE 
(W X S) 


MAX WT SCORE 

(w x Sfflax ) 


WT SCORE/MAX WT SCORE 




- date of last medical 

- physician information 

- _alth care mmber 

- departure inforration 

- date 

- reason 

- minimum of 6 user defined fields 














Instructor Information 


5 


5 


25 








- instructor code 

- name 

*- address 

- telephone 

- social insurance number 

- language of instruction 

- certificate number 

- courses taught 

- minimum of 6 user defined fields 














Course information 


15 


2 


30 








- course code (5 character alpha-numeric) 

- description 

- pre-and co-requisites (mi iraum of 4) 

- must handle"and*7"or"situation 

- course type 

- language of instruction 

- course accreditation 

- credit value (2 digits) 

- pass/ fail mark 

- grade 










i 




TOTAL Detailed Data I tew 


45 


11/30 


155 


450 


.33 


156 


i 










.157 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



WEIGHT 
(W) 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



15^ 



ERIC- 



Reports/Inquiries 

All reports and inquiries should be avail- 
able for all or a specified range of 
records, in various sort orders. 

- class lists 

- homeroom lists 

- student name labels 

- student address labels 

- parent address labels 

- student I.D. cards 

- student data (alphabetical or numerical 
order) 

- parent data (alphabetical or numerical 
order) 

- instructor data (alphabetical or numer- 
ical order) 

- course data 

- student phone list 

- student name list 

- student grade list 

- feeder school list 

- locker information list 

- student population by instruction t/pe 

- fee sheets 

The system should allow production of 
user-defined reports/ inquiries using 
available data. 

TOTAL Reports/Inquiries 

TOTAL SCHOOL RECORDS 



25 



50 



25 



90 



50 



21/70 



325 



250 



900 



.36 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEICHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX UT SCORE 


WT SCORE/MAX WT SCORE 




SCHEDULING 














Detailed Data Items 














- Course code 

— Course section 














Manual scheduling (Arena Scheduling) 


7 


6 


42 








Pre-scheduling 
Course Requests 














manual entry 


5 


2 


10 








automated entry 


9 


0 


0 








- allow student to specify mandatory/ 
c oipulsory courses, 

- preferred courses, preferred 
alternatives, etc. 

- allow student to specify preferred 
section, semester, or instructor 














Edit and validation of course requests 


7 


4 


28 








- checking of pre- and co-requisites in 
the current students' requests ?j well 
as history files 

- capability to override pre- and co- 
requisites 

- capability to complete pre-requisite 
checking for students from other 
District schools. 

Pre— scheduling reports 


9 


4 


36 






160 


- potential conflict matrix — for all 
or a specified range of courses. 
Additional selection criteria may be 










1C I 



EVALUATION 
FACTOR 



CRITERIA rrois 



WEIGHT 
(W) 



SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



based on the number of requests or the 
number of sections. 

- course tally 

- students with no requests 

- student course request list 

- min/max request li 

- min/max credit list 

- verification tickets 

- arena scheduling labels 

- stud nts missing compulsory courses 

- students requesting specific course or 
group of courses 

Master scheuule builder 



Capability to build a master schedule 





n.*nuaily 


6 


8 


48 




automatically 


9 


0 


0 




Capability of handling p variety of 
Scheduling units 


9 




36 


1 


- full y*_3r 

- semester 

- trimester 

- quartermester 

- 6 ;^eek unit 

- any combination of the above 










User defined timetable rotation/tumble 


10 


4 


4 J 




Flexible number of periods per day 


10 


6 


60 




Capability to specify exclusive male or 
female sections 


5 


7 


35 




Capabilit}' to *naintain current and future 
ye^r/semester master schedules 


8 


5 















EVALUATION 


CRITERIA ITEMS 


WEIGHT 


FACTOR 




(W) 




Scheduling Process 






User defined scheduling sequence 


6 




- low grades first 






- high grades first 






- A to Z 






- Z to A 






Unscheduling of no-sho^ Withdrawals 


5 




Scheduling of individual student ot small 






groups of students 


6 




Capability to reset all students or 






partially scheduled student? 


8 




Capability to Jock scheduling assignments 






tor all students or a group of students 


8 




Restart capability 


8 




Course weighling/sen^stet balancing 






(ensure even course load for students) 


8 




Blocking of courses 


7 




Section balancing 


8 




Class balancing (riles-females) 


4 




Capability to keeD scheduling nnpn affpr 






school start while starting to use the 






attendance module 






Scheduling Reports/inquiries 


10 




- student timetables — grid and list 






format 






- instructor timetables — griJ nnd list 






fornct 






- room timetables — grid and list format 






- master schedule 






- student scheduling conflicto 






students partially scheduled 






- unas signed time 






SCORE 
(S) 



WEIGHTED SCORE 
(W X S) 



10 



8_ 
~4 



24 



25 



AO 



80 
64 

40 
56 
64 
16 

0 
30 



MAX WT SCOPE 



WT SCORE/MAX WT SCORE 



0 
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FVA1 ITATTHM 1 

FACTOR 




ur j cut 

(W) 


(S) 


(W X S) 


may Lfr orr»Dc 
< W X W 


ltt crr^DT? /may tfr crnor 


















Homeroom grouping for core subjects 


9 


4 


36 








1 hi 1 f f" 17 ni C hon 1 1 1 { T"l fT nnu r*r\t iron {n 

udpauiii iy uj> ov-iicuuj. iii^ any course in 
any combination and number of time 
dptI ods 


in 


A 


An 








TOTAL SCHEDULING 


200 


118/2«) 


890 


2000 


.45 




STUDENT ATTENDANCE 














Kntrv of AffpnHanPP Tiata 

JUL i T ul flLLCllUO L/(l Lu 














manual entry 


5 


5 


25 










Q 


u 


U 








Multiple user-defined absence types 


8 


6 


48 








Capability to record attendance data at 
various intervals* 


10 


6 


60 








- daily 

- twic* per day 

- period by period 

oUUJCLL Uj £>UU JCLL 














Attendance history 


8 


5 


40 








- at least ten days detail 

t-liUIml J.O L i. V" LULciXb 














Attendance r-ports/inquiries 


10 


3 


30 






itio 


- student by class 

- student by subject 

- student by period 






_i 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 

(w) 


SCORE 

(s) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE WT 

(w x w 


SCORE/MAX W£ SCORE 




- I.cmeroora attendance 

- daily summary 

- v T eeklv summary 

- ronthly summary 

- miltiple absence 

- Cc ^ability to produce unexcused 
absence report for the current day 
within 30 minutes 

- the system should allow user defined 
reports/inquiries using available data 














TOTAL ATTENDANCE 


50 


25/60 


203 


500 






STUDENT MARKS 














Entry of marks data 














manual 


5 


7 


35 








automated 


9 


0 


0 








Marks data 


10 


6 


60 








- minimum of 4 term marks plus final mark 

- letter or percentage grades 














Student Exams 


6 


4 


24 








Exam timetable builder 














- automated 
" manual 














Exnm Reports/lrquiries 












■ 


- potential exam conflict matrix 

- exam schedules 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



WEIGHT SCORE 
(W) (S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 



WT SCORE/MAX WT SCORE 



(EASE OF 
USE 



ERIC 



Reports/Inquiries 

proof list 
report cards 

- narks data 

- final mark, calculated according to 
user-defined formula attenandance data 

- class averages 

- honour lists 

- potential failure lists 

- graduation list 

TOTAL STUDENT MARKS 



UTILITY FUNCTIONS 

Backup/Restore 
Security/ Controls 
TOTAL UTILITY FUNCTIONS 

GRAND TOTAL, PRODUCT SCOPE AND IUNCTION 



- flexibility 

- modular, table drivei 

- help facilities 

- menu driven 



GRAND TOTAL, EASE OF USE 



10 



40 



12 



20 



400 



60 



60 



22/50 



6/20 



197/460 



50 



169 



72_ 
_0_ 
72 



1659 



300 



300 



400 



200 



4000 



[ 



600 



.42 



.36 



.41 



.5 



EVALUATION 
FACTOR 



TECHNICAL 
CONSIDERATIONS 



SUPPORT & 
SERVICES 
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CRITERIA ITEMS 



hardware 

system software environment 

- operating system 

- utilities 

- database management /system 
internals/files 

- networking capabilities 

- user hooks 

- modularity of the system 



GRAND TOTAu, TECHNICAL CONSIDERATIONS 



- local versus where/how far 

- package support aitd services 

- software support, custom 
modifications 

- documentation 

- user guide, application system, 
Procedural, operations guide , 
file layouts 

- training 

- applications system, operational 
(DP), availability schedule, format, 
location, prerequisites 

- implementation 

- training 

- initialization (conversion , file set- 
up, output forms) 

~ implementation plan 



GRAND TOTAL, SUPPORT 6 SERVICES 



WEIGHT 
(W) 



80 



80 



SCORE 
(S) 



70 



70 



WEIGH 1 SCORE 
(W A 3) 



240 



240 



140 



140 



MAX WT SCORE 
(W X 



S max) 



800 



WT SCORE/MAX WT SCORE 



.3 



700 
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FACTOR 




(W) 


(S) 


1 

(W X S) 


MAY L7T CmDF 

< H X W 






PRODUCT 
QUALIFICATIONS 




80 


4 


320 












- package background 

- reliability 

- rurrent development status 

- number of installations 

- product development plans 

- release concept, portability, 

\TOT"t" "1 /^O 1 "f t~ \7 
VCL LlLalllY 


















GRAND r jX)TAL, PRODUCT QUALIFICATIONS 




80 


4 




320 






80U 






.4 






VENDOR 




70 


4 


280 
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- Corporate information 

- background and history 

- financial performance 

- employee base 

- Market volatility and vendor stability 

- References 

- Contractual Terms 

- maintenance 

- warranty 

- nur p r r Vi i n ffcrHffi 

UWl iC L Ol UL LJ L -L-K ^ * 

- discount structure/price limit 






i 












GRAND TOTAL, VENDOR 




70 


4 




280 






700 






.4 
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Observations 



The following comments and observations arj offered in support of the 
quantitative evaluation of SIRS. 

(A) Product Scon_ and Function 



Pre-Re^istration: - is goo 1 and fast. 

Detailed Data Itens: - the oackage provides some of th key data items 

but does Miss some of the most essential fields 
such as Fa^der School, previous schools, 
teligious denomination and certain medical 
information. Mosr importantly, there are no 
user defined fields. It is vital that schools 
have the flexibility to define tl ^ir own 
student, demographi : data. 

- Instructor information is basic with again, no 
user defined fields. Course information is 
very limited and really falls short of minimum 
criteria. 

Reports/Inquiries : - Inquiry facilities are virtually non- 

existent. Reporting is v^ry li.nited and very 
poor in quality. During the test, a number of 
haud-written reports were needed to track the 
set up and development of data. No facilities 
<?xisc for ihe generation of user-def ired 
reoortj, there is no *eport generator anc no 
da* * f Lie layouts. 

Scheduling; - Basic dai_a entry and manual scheduling (ARENA) 

are acceptable and workable. Manual entry of 
r Tse requests is confusing and tedious, 
lucre are no automated entry fac 'it "e<5. 

- Editing and validation of course requests 
involves a 2 stage batch process. There is no 
interaction with the user and the results are 
poo> ly presented and ca r be confusing. 

- Pre-scheduling reports are available, bvt 
fairly limited in their usefulness. Several of 
the reports, especially exception reports, in 
the selection criteria are not available. 

~ The Master schedule builder produces a good 
manual schedule; tnere are no automatic 
schedule builder facilities. Some scheduling 
un,*ts can not be handxeJ. Mrst essential 
features t such as rctaciuii/tumble and different 
numbers of periods per day can be hand^J. 
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- The scheduling process is fairly poor it. die 
areas of interaction and user parameters - 
these were "stripped" ou*" for the IBM 
implementation. Schedul ing of small groups of 
students is impossible as is the ability to 
tceep scheduling open after the start of the 
school year. Most other features such as 
restart capability, blocking and section 
balanci ig are handled well but with no user 
control. 

~ Scheduling epoits are poor; inquiries arc 

virtually non-existent. Student timetables and 
"partials'' can be generated. After running che 
Toad/Prirt batch process, cl iss lists can be 
generated. 

- Homeroom grouping is han<J ^d In a fashion 
axthough there is no rser control over the 
actual students within each group. It is the 
fact that Junior High schools have large core 

- object groupings that causes thu. scheduler to 
produce such poor results. Affft filling the 
timecable from optional courses (which 1 ave 
fewer sec:ions offered) 1l is very difficult to 
fit the It or 16 period core groups- into the 
timetable. 

Student Atten dar ce: - Attendance software v?.s tested in outline and 

was found to be acceptable in most areas* 

- Manual en'ry is fairly difficult due to the 
absence of key reports; there are no automated 
(scanner) facilities. There is a limited 
number or atcence codes and attendance can be 
recorded at different intervals (mo^t junior 
high schools *ecord attendance at the half day 
level) • 

- Reporting is fairly poor with some key reports 
missi lfi and layout on tne working reports very 
pc.or. Ih^re are no user defined reporting 
facilities* an important requirement in this 
area. 

Student Marks: - This are/ was also z~.Zt.ed in outline and was 

found to be reasonable ir. most functions. 

- Manual entry of data was straight forward but 
tedious; th^re is no automat 2d f aciiicy. Most 
marks data requirements were met, but student 
exam facilities were poor. 
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- Reports and Inauirics were acceptable tnis 
area, but certain key reports v ;pre mijsing an i 
inquiry facilities were minimal. 

Utility Functions: Backup and Hi tore ar: handled through the i£M PC. 

DOS c .erating system utilities of trie same 
name: SIRS does not havo its own backup/ restore 
software • 

There were no security controls in :he software 
which was purchased for evaluation, although 
there may be facilities of this kind on 'he 
multi-user NCR or ALTOS systems . 

(B) Ease of Use ihe package is not easy t_ v use despite the menu 

facilities and the modularity of the coda (built 
as a series of COBOL objects). 

Several menus lead to the wrong p,.\>gram or 
facility; there are no "help" facilities; the 
system is not flexible cr adaptable. 

(C) Technical ConsiderKtions : 

Tit; SIRS system lutis on ire IBM PC rang* oJ 
microcomputers but does not take ad\ antage of 
nuny of the features of the machine - fast screen 
painting, spooling, disk caching to memory, etc. 

The RM COBOL system runs fairly well but slowly 
under the PC DOS operating svg^c^: it doesn't 
appear to have "hooks" into the operating svste^i 
directly. 

Stardard PC DOS Utilities were used thioughout 
the test and it would s^em that the SIRS package 
has no utilities. 

No technical information oi ptogrammer 
information was provided and we can only surmise 
that the database management; system is a standard 
facility provided by Ryan McFar Lam 1 (the vendor 
of RM COBOL). 

Networking (using a Local Ar^a Network) w^s not 
feasible and there were no user hooks provided 
for the Junior High system. (Two RM COBOL 
programs were provided for the Senior Hi£h system 
for the loading of data). The system was modular 
insofar that it wrs compo-ed of a number of CG30I 
modules. 
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(D) Support and Service 



There was basicall> no real support and minimal 
training. Training consisted of a 1 hour session 
where the softwj^e was copied onto the IBM XT and 
started up. 

One modification was produced - to allow up to 9 
core subjects to be grouped - but thero were no 
gtneral release plans announced. 

Documentation is poor and untidy. There is no 
"roadraap" and no startup list. A school 
administrator would not be able to use the system 
with tne documentation pro\ided. 

There were no training or setup plans provided by 
the vendor. Similarly there was no 
implementation plan -yjith the result that on a 
number of occasions datp was punched into the 
wrong database file. 

(E) Pioduct Oiialif icaticms: 

The package has been developed and maintained 
since 1979 and is faitly stable with sone new 
releases planned. The vendor plans to recode 
some of the programs in "C" and improve "core 
grouping". There are only 7 installations, not 
all of which are junior high schools, uj 4 ng the 
package (some of these installations, for 
example, Rocky Mountain House, have not yet used 
the scheduler) • 

(¥) Vendor: The vendor is locally based, in St. Albert, and 

has installed a few SIRS systems within 
Alberta. Vendor stability and corporate 
information is poor and experiences tended to be 
neutral. It is a small, local company and would 
not be able to support multiple problems at 
differenc sites. 
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6.1.3 System Performance, Strengths and Weaknesses - SIRS (MIG) 
Key Performance Indicators 

School Test Site Parameter IBM PC/XT 



Scheduler - Time 




? 


-34 


hours 


Scheduler - Performance 


8 


1% 




Scheduler - Expected 


Perf . 


99% 




Tine tables 




2 


:00 


hours 


Conflict Matrix 




1 


:00 


hour 


Course Tally 




0 


:32 


hour 


Master Schedule 




2 


:15 


hour 


Class Lists 




2 


:00 


hours 


Attendance Registers 


and Rpt 


3 


:30 


hours 


Marks Registers 






42 


rain. 


Student Registers 




2 


:00 


hours 



Steele Heights Junior High School: 646 students 
(All times ere in hours :rainutes) 
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System Strengths: 



Mainframe compatibili ty : 



User friendly: 
Spooler function: 



Core subject grouping: 



Ryan McFarland, the producers of IW COBOL, have 
developed a VM/CMS version of this package 
which works on the IBM 4341 mainframe. Thus, 
minor modifications to COBOL source and JCL 
should result in a mainframe version of SIRS. 

Tha package? is fairly interactive and at no 
lime did it give system errors or "abends" . 

While reports cannot be spooled to the printer 
(in caramon with other micro-computer packages), 
there is an option to spool to disk for later 
printing. 

SIRS provides facilities for grouping core 
subjects together, thus forcing common sections 
for groups of students. This is essential in 
junior high Schools. 



System Weaknesses: 
Reports : 



Database files: 



Logical steps: 



Tu£ reports are very poor and not well 
dei. ib<?d in the menu programs. In most cases, 
we ave ui.able to get the information required 
to verify data and proceed to the next stage of 
d velopment. Hand written forms had to be 
designed and updated constantly. 

SIRS appears to hold two sets of files with 
some very confusing results. Op two occasions 
fairly large amounts of data were keyed into 
the wrong file and had to be r°-enteted. 

There were too steps required in designing 

the Class Mastei Schedule and running the 
Scheduler. Each step was separated by long 
periods of waiting to enter the next command 
due to the absence of batcL or indirect command 
file control. 



Scheduling results: 



The Load/Print and Simulation phases uid net 
provide clues as to what had caused classes to 
conflict. In genoral , thur<: were fev audit 
trails or dotal j.o of e rors. 



Documentation : 



No puidelines explaining th? sequence of 
operations. 



1 8 1 

(108) 



Specific Systems Problems; 



No Section "forcing* : 



for example student #7107 requested two 
sections of the Maths Support course but there 
was no facility for specifying a particular 
section or sections. 



Current Grader 



All students were entered as currently 
operating within the school. When the 
Scheduler Student/Request edit list was run, 
all students were flagged as being in the wrong 
grade and every record required editing. 



No "Edit/Error only 
listings 



Student Requests: and Harks/Attendance data 
entry did not have reports describing errors 
only. 



Request File: 



No summary listing of the Request File. 



6.2 Evaluation of TSS 

6.2.1 Testing Environment and Conditions 

The same school, Steele Heights Junior High School, that was ULied for t>e 
MIG SIRS tests was also used to pilot test The School System. The school 
has 646 students enrolled in grades 7,8 and 9. It uses a strict 4 day, 6 
period rotation schedule and operates attendance at the half-day 
reporting period level. 

The Sc^ol System was tested on an IBM PC/XT with 10 Mbytes hard disc 
storage 512 kb memory, an Okidata Microline 84 printer operating at 
about r ) c.p.s and a 360 kb diskette drive. The system supports two or 
more users but this feature was not tested. 

Data ^ntry was entirely by keyboard for this rest although data loading 
facilities from the mainframe computer can be accomplished using the 
virtual scan input mechanism. 

6.2.2 Evaluation Results and Observations 

The following tables show the outcone of the quantitative evaluatior of 
TSS against the d^taJled evaluation criteria. 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


1 

WEIGHT 
(W) 


SCuRE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX WT SCORE Wr 
< w X S max> 


SCORE/MAX WT SCORE 




PRODUCT 
SOCFE & 
FUNCTION 


SCHOOL KEOOKDS 

Pre-Registration/Enrollment 
Create student record 


13 


6 


90 


— I 








- school student I.D. 

- last name 

- middle name 

- first name 

- birthdate 

- current grade 

- sex 

- feeder school 

- home address 

Registration conf irroation notice 


3 


5 


15 






o 




Feeder school confirmation notice 


2 


4 


8 










TOTAL Pi^Registration/EnrolLnent 


20 


15/30 


113 


200 


.56 




! 


Detailed Pata Items 
















Student information 


25 


9 


225 






lb 
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school student I.D. 

- District student I.D. 

- Mberua Education st.^en: I.D. 

- last name 

- middle name 

- first name 

- birthdate 

- current grade 

- cex 

- feeder school 

- home address 

- telephone number 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



- emergency contact 

- name 

- telephone 

- entry information 

- entry date 

- regirtration code 

- withdrawal code 

- previous schools (2) 

- homeroom instruction 
~ counsellor 

- parent /guardian information (up tc 4) 

- name 

- address 

- telephone (home and business) 

- relationship 

- occupation 

~ locker information 

- number 

- combination 

- student indebtedness 

~ religious denomination 
~ program type 

- number of credits earned 

- this school 

- other scnools 

- academic history 

- travel information 

- method 

- distance 

- bus pass information 

- parking information 

- driver's licence 

- licence plate 

- parking space 

- meJi .al irformation 

- disabilities/behaviours 

- medications 

- allergies 



WEIGHT 
(W) 



SCOR£ 
(S) 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 

< w x w 



1 



WT SCORE/MAX WT SCORE 



EVALUATION 
F 7T0R 


CRITERIA ITEMS 


1 

WEIGHT 
(W) 


SCORE 

(?) 


WLIGHTED sCORE 
(W X S) 


MAX wl SCORE 


wT SCORE/ MAa wT SCORE 


i 


- date of last medical 

- physician information 

- health care number 

- departure information 

- date 

- reason 

- minimum of ( user defined fields 














Instructor Information 


5_ 


5 


25 








- instructor code 

- name 

- address 

- telephone 

- social insurance numbc~ 

- language of instruction 

- certificate number 

- courses t£/ght 

- minimum or 0 user defined fields 














Course information 


15 


9 


135 








- course code (5 character alpha -numeric) 

- description 

- pre-and co-requisites (minimum of 4) 

- must handle"and"/ t, or ,, situation 

- course type 

- language of instruction 

- course accreditation 

- credit value (2 digits) 

- pass/fail m^k 

- prade 














TOTAL Detailed Data I tew 


45 


23/30 


385 


450 


.85 


lav 
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EVALUATION 
FACTOR 
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CRITERIA ITEMS 



Reports/ Inquiries 

All reports and inquiries should be avail- 
able for all or a specified range of 
records, in various sort orders. 

- class lists 

- homeroom lisfs 

- student name labels 

- student address labels 

- parent address labels 

- student I.D. c*~ds 

- student data ( phabetical or numerical 
order) 

- parent data (alphabetical or numerical 
order) 

- instructor data (alphabetical or numer- 
ical order) 

- course data 

- student phone list 

- student name list 

- student grade list 

- feeder school list 

- loc ?r information list 

- student population by instruction type 

- fee sheets 

The system should aJlow production of 
user-defined reports/inquiries using 
available data. 

TOTAL Reports/Inquiries 

TOTAL SCHOOL RECORDS 



T¥5 



WEIGHT 
(W) 



25 



SCORE 
(S) 



25 



90 



45/70 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 

< M x w 



175 



175 



673 



250 



WT SCORE/MAX WT SCORE 



.7 



900 



.7* 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX Wi SCORE 
< W X W 


WT SCORE/MAX WT SCORE 




SCHEDULING 














Detailed Data Items 














- Course code 

- Course section 










! 




Manual scheduling (Arena Scheduling) 


7 


9 


63 








Pre-scheduling 














Course Requests 














manual entry 


5 


9 


45 








automated entry 


9 


10 


90 








- allow student to specify mandatory/ 
compulsory courses, 

- preferred courses, preferred 
alternatives, etc. 

- allow student to specify preferred 
section, semester, or instructor 














Edit and validation of course requests 


7 


9 


63 








- checking of pre- and co-requisites in 
the current students 1 requests as well 
as history files 

- capability to override pre- and co- 
requisites 

- capability to complete pre-requisite 
checking for students from other 
District schools. 

Pre-scheduling reports 


9 


8 


72 






191 


- potential conflict matrix — for all 
or a specified range of courses. 
Additional selection criteria may be 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



based on the number of requests or the 
number of sections, 

- course tally 

- students with no requests 

- student course request list 

- min/max request list 

- min/max credit list 

- verification tickets 

- arena scheduling labels 

- s;udents missing compulsory courses 

- students requesting specific course or 
group of courses 

Master schedule builder 

Capability to build a master schedule 

manually 

automatically 
Capability of handling a variety of 
Scheduling units 

- full year 

- semester 

- trimester 

- quartermester 

- 6 week unit 

- any combination of rhe above 

User defined timetable rotation/tumble 
Flexible number of periods per day 
Capability to specify exclusive male or 
female sections 

Capability to maintain current and future 
year/semester master schedules 



WEIGHT 
(W) 



SCORE 
(S) 



10 


10 


100 


10 


10 


100 


5 


8 


40 


8 


5 


40 



WEIGHTED SCORE 
(W X S) 



MAX WT SCORE 



-+- 



54 



63 



WT SCORE/MAX WT SCORE 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


"I 

J 

MAX WT SCORE 1 

< M x w 


WT SCORE/MAX WT SCORE 





Scheduling Process 














User defined scheduling sequence 


6 


5 


30 








- low grades first 

- high grades first 

- A to Z 

- Z to A 

Unscheduling of no-shows/wit. drawals 


5 


10 


50 








Scheduling of individual student or small 
groups of students 


6 


8 


48 








Capability to reset all students or 
partially scheduled ?tudent c 


8 


10 


80 








Capability to lock scheduling assignments 
for all students or a group of students 


8 


10 


80 








Restart capability 


8 


0 


0~~ 








Course weight ing/semester balancing 
(ensure even course load for students) 


8 


7 


56 








Blocking of courses 


7 


10 


70 








Section balancing 


8 


8 


64 








Class balancing (males-females) 


4 


"7 


28 






i 


Capability to keep scheduling open after 
school start while starting to use the 
attendance module 


9 


9 


81 








Scheduling Reports/Inquiries 


10 


9 


90 






J9b 


- student timetables — grid and list 
format 

- instructor timetables — grid and list 
format 

- room timetables — grid and list format 

- master schedule 

- student scheduling conflicts 

- students partially scheduled 

uiiaooigiicu L x l lie 

! 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



Junior High Scheduling Requirements 

Homeroom grouping for core subjects 
Capability of scheduling any course in 
any combination and number of time 
periods 

TOTAL SCHEDULING 



STUDENT ATTENDANCE 

Entry of At t endance Data 

manual entry 
automated entry 

Multiple user-defined absence types 

Cappbility to record attendance data at 
various intervals 

- daily 

- twice per day 

- period by period 

- subject by subject 

Attendance history 

- at least ten days detail 

- cummulative totals 

Attendance reports/inquiries 

- student by class 

- student by subject 

- student by period 
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WEIGHT 

tw) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


KAX WT SCORE 

(w X S^J 
max 


WT SCORE/MAX WT SCORE 


9 


9 


81 






10 


10 


_100 






200 


206 


'^38 


2000 


.79 


5 


9 


45 






9 


10 


90 






8 


10 


80 






10 


10 


100 






8 


9 


72 






10 


6 


60 
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FACTOP 




(W) 


(S) 


(W X s) 


MAX OT S."0RF 
< W X S max> 


WT SCORE/MAX OT SCORE 




- homeroom attendance 

- daily summary 

- weekly summary 

- monthly summary 

- multiple absence 

- capability to produce unexcused 
absence report for the current day 
within 30 minutes 

- the system should allow user defined 

LcJa L Lb/ l[l(|UlLlcb Ublilg; aValliDlc Qala 














TOTAL ATTENDANCE 


50 


54/60 


447 


500 


.89 




STJDENT MARKS 














Fr.fTV r»f~ marlrc data 














manual 


5 


9 


4f- 








automat pH 


9 


10 










Marks data 


10 


10 


100 








- minimum of 4 term marks plus final mark 

— Ipffpr nT* nprf*pnt* Acrp irrAHpQ 

1CLLCL v/L J^C 1. LCL1LCIKC cL OUC9 














Student Exams 


6_ 


0 


0 








Exam timetable builder 














- automated 

- manual 














Exam Reports/ Inquiries 
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- potential exam conflict matrix 

- exam ochedules 










2'0 



EVALUATION 
FACTOR 



EASE OF 
USE 



o 201 



CRITERIA ITEMS 



Reports/Inquiries 

p oof list 
report cards 

- marks data 

- final mark, calculated according to 
user-defined formula attenandance data 

- class averages 

- honour lists 

- potential failure lists 

- graduation list 

TOTAL STDDOTT MARKS 



DflLITY FUNCTIONS 

Backup/Restore 
Security/ Controls 
TOTAL UTILITY FUNCTIONS 

GRAND TOTAL, PRODUCT SCOPE AND FUNCTION 



flexibility 
modular, table driven 
help facilities 
menu driven 



GRAND TOTAL, EASE OF USE 



WEIGHT 
(W) 



10 



SCORE 
(S) 



40 



12 



20 



400 



60 



35/50 



8/20 



WEIGHTED SCORE 
(W X S) 



348/460 



3 



60 



295 



96 



96 



3099 



540 



540 



max vrr SCORE 



400 



200 



4000 



600 



WT score/max wt score 



.74 



.48 



.7747 



.9 



202 



EVALUATION 
FACTOR 



TECHNICAL 
CONSIDERATIONS 



O 



SUPPORT & 
SERVICES 
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CRITERIA ITEMS 



hardware 

system software environment 

- operating system 

- utilities 

- database management/system 
internals/files 

- netvrorking capabilities 

- user hooks 

- modularity of the system 



GRAND TOTAL, TECHNICAL CONSIDERATIONS 



local versus where/how far 
package support and services 

- software support, custom 
modifications 

documentation 

- user guide , application system, 
procedural, operations guide, 
file layouts 

training 

- applications system, operational 
(DP), availability schedule, format, 
location, prerequisites 

implementation 

- training 

- initialization (conversion, file set- 
up, output forms) 

- implementation plan 



GRAND TOTAL, SUPPORT & SERVICES 



WEIGHT 
(W) 



80 



80 



70 



70 



SCORE 
(S) 



10 



10 



WEIGHTED SCORE 
(W X S) 



800 



800 



630 



630 



MAX WT SCORE 



800 



WT SCORE/MAX WT SCORE 



700 



1.0 



•9 





EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


SCORE 
(S) 


WEIGHTED SCORE 
(W X S) 


MAX vrr SCORE 
max 7 


WT SCORE/MAX WT SCORE 




PRODUCT 
QtWULFlCATlOHS 




80 


9 


>20 














- package background 

- reliability 

- current development status 

- number of installations 

- product development plans 

- release concept, portability, 
vertically 














i 






GRAND TOTAL, PRODUCT QUALIFICATIONS 


80 


9 


720 






800 






.9 








VENDOR 




70 


8 


560 
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- Corporate information 

- background and history 

- financial performance 

- employee base 

- Market volatility and vendor stability 

- References 

- Contractual Terms 

- maintenance 

- warranty 

- ownership rights 

- discount structure/price limit 




















GRAND TOTAL, VENDOR 


70 


8 


560 




' 700 






.8 




0 


205 
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Observations 

(A) Product Scope and Function 

Junior High Requirements The system allows homeroom grouping using 

course relationships. 

Enrolment of students in a group of classes is 
allowed, for example, core classes. This is 
achieved very quickly using the auto course 
selection option. 

Flexible timetable rotation allows scheduling 
of any class in any combination of periods. 

All other criteria observations are listed in section 4.2.2 for Senior 
High schools and, In this case, are pertinent for the junior high school 
evaluation. 

£•2.3 System Performance, Strengths and Weaknesses - TSS (Columbia) 

Key Performance Indicators 

School Test Site Parameter IBM PC/XT 

Steele Heights Scheduler - Time 1:35 hours 

(0:40 hours) 

Scheduler - Performance 100% 

Scheduler - Expected Perf. 100% 

Timetables 7:30 hours 

Conflict Matrix 1:50 hours 

( extrapolated) 

Course Tally 0:15 hour 

Master Schedule 0;1S hour 

Class Lists 3:30 hours 

Attendance Registers 3:30 hours 

Marks Registers 3:30 hours 

Student Registers 1:30 hours 



Steele Heights Junior High School: 646 students 
(All times are in hours: minutes) 
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S ystem Strengths : - capability of handling homeroom grouping 

- ability to schedule any course in any 
combination and/or number of periods 

- capability of handling with ease any rotation 
tumble for any number of periods. 

- Also see section 4.2.3 

System Weaknesses : (see Senior High School descriptions in Section 4.2.3) 

7.0 COMPARATIVE EVALUATION OF SIMS - JUNIOR HIGH SCHOOL PERSPECTIVE 

7 . 1 Comparison Summary and Review of SIMS Evaluation Data 

The following tables show the quantitative evaluation data for the two 
microcomputer based SIMS which were evaluated in detail at the junior 
high school level. As was the case of the senior high school 
perspective, the Detailed Scoring Comparison Form has been used to 
display the data. A third system, the School Administration System by 
SIERRA was also e/aluated at the junior high school level. This 
particular system, which runs on the Digital Equipment Corporation's 
(DEC) VAX family of computers, was evaluated using a VAX 11/725 
minicomputer. In view of this, the outcomes of the evaluation are 
included within a separate report. It should be noted, however, that 
because the system runs on a smaller more affordable VAX computer called 
the MICROVAX, the results of the evaluation may be of interest to schools 
seeking a microcomputer bas^d solution. 



2\)8 

9 (123) 

ERIC 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



PRODUCT 
SCOPE & 
FUNCTION 



SCHOOL RECORDS 

Pre- Regis t ration/Enrollment 

Create student record 

Registration confirmation notice 
Feeder school confirmation notice 

TOTAL Pre-Registration/Enrollaent 

Detailed Data Items 
Student information 
Instructor Information 
Course info-mation 

TOTAL Detailed Data Items 
Reports /Inquiries 
TOTAL Reports/Inquiries 
TOTAL SCHOOL RECORDS 
SCHEDULING 

Manual scheduling (Arena Scheduling) 



t. Ui 



TSS 

WEIGHT SCORE WEIGHTED 
SCORE 

(W) (S) (W X S) 



SIRS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 



15 



90 



120 



15 



20 



15/30 



113 



8/:o 



120 



25 



225 



.'00 



25 



25 



15 



135 



30 



45 



23/30 



385 



1J/30 



155 



25 
25 



90 



7/10 
45/70 



175 
175 

673 



2/10 
21/70 



50 
50 



325 



63 



42 
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;V ALU ATI ON 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


TSS 

SCORE HEICHTED 
SCORE 
(S) (W X S) 


SIRS 

SCORE WEICHTED 
5 CO RE 
(S) (W X S) 




Pre-scheduling 














Course Requests 














manual entry 


5 


9 


45 


2 


10 




automated entry 


9 


10 


90 


0 


0 




Edit and validation of course requests 


7 


Q 

y 


63 


/ 

k 


28 




Pre-scheduling reports 


9 


8 


72 


u 


36 




TOTAL r re— Scheduling 


30 




Z/O 


10/40 


74 




Master schedule builder 














Capability to build a master scheduler 
manually 


6 


9 


54 


8 


48 




automatically 


9 


o 


0 


0 


0 




Capability of handling a variety of 
scheduling units 


9 


7 


63 


4 


36 




User defined timetable rotation/ triable 


10 


10 


100 


4 


40 




Flexible number of periods per day 


10 


10 


100 


6 


60 




Capability to specify exclusive male or 
female sections 


5 


o 
0 


40 


1 


35 




Capability to maintain current and future 
year/semester master schedules 


8 


5 


40 


5 


40 




total Master Schedule Builder 


57 


AS/70 




397 


34/70 


259 




Scheduling Process 














User defined schedule ng sequence 


6 


5 


30 


4 


24 




Jnschedul tng of no-shows /withdrawal 8 


5 


10 


50 


5 


25 


211 
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EVALUATION 
FACTOR 


CRITERIA If EMS 


WEIGHT 
(V) 


TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


SIRS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 




Scheduling of individual student or snail 
groups of students 


6 


8 


48 


0 


0 




Capability tr reset all students or 
partially scheduled students 


8 


10 


80 


5 


40 




r --pability to lock scheduling assignments 
for all students or a group of students 


8 


\0 


80 


10 


80 




Restart capability 


8 


0 


0 


8 


64 




Course weighting/semester balancing (ensure 
e^en course load for students) 


8 


7 


56 


5 


40 




Blocking of courses 


7 


10 


70 


8 


56 




Section balancing 


,3 


8 


64 


8 


64 




Class balancing (males-females) 


4 


7 


28 


4 


16 




Capability to keep scheduling open af^er 
school start while starting to use the 
attendance module 


9 




9 


81 


0 


0 




TOTAL Scheduling Process 


77 


84/110 


.S87 


57/110 


409 




Scheduling Report j/ Inquiries 


10 


9 


90 


3 


30 




Junior Hig.i Scheduling Requirements 














Homeroom grouping for core subjects 




9 


81 


4 


36 




Capability of scheJ'jling any course in any 
combination and nuitber of time periods 


10 


10 


100 


4 


40 




TOTAL SCHEDULING 


200 


206/260 


1588 


118/260 


890 


















Entry of Attendance Data 














manual entry 


5 


9 


45 


5 


25 




automated entry 


9 


10 


90 


0 


0 

















EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


SIRS 

SCORE WEIGHTED 
SCORt 
(S) (W X 3) 




Multiple user-defined absence ^ypes 


8 


10 


80 


6 


48 




Capability to record attendance data at 
various intervals 


10 


i n 
lU 


100 


6 


60 




Attendance history 


8 


9 


72 


5 


40 




Attendance reports/inquiries 


10 


6 


60 


3 


30 




TOTAL ATTENDANCE 


50 


54/60 


447 


25/60 


203 




STUDENT MARKS 














Entry of marks data 














manual 


5 


9 


45 


7 


35 




automated 


9 


10 


90 


0 


0 




Marks data 


10 


10 


100 


6 


60 




Student Exams 


6 


0 


0 


4 


24 




Exam timetable builder 
Exam Reports/Inquiries 














Reports/Inquiries 


10 


6 


60 


5 


50 




TOTAL STUDENT MARKS 


40 


35/50 


295 


22/50 


169 


215 

o 












21 
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EVALUATION 
FACTOR 



EASE OF 
USE 



CRITERIA ITEMS 



UTILITY FUNCTIONS 
Backup/Restore 
Security/Controls 
TOTAL UTILITY FUNCTION* 

GRAND TOTAL, PRODUCT SCOPE AND FUNCTION 



TECHNICAL 
CONSIDERATION 



SUPPORT & 
SERVICES 
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GRAND TOTAL t EASE OF USE 



GRAND TOTAL, TECHNICAL CONSIDEkATIONS 



GRAND TOfAL, SUPPORT & SERVICES 



WEIGHT 
(W) 


TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


SIRS 

SCORE WEIGHTED 
(S) (W X S) 


12 


a 




0 


/ Z 


8 


0 


0 


0 


0 


20 


8/20 


56 


6/20 


72 












400 


348/460 


2099 


192/460 


1659 


60 


9 






'inn 












60 


9/10 


*40 


5/10 


300 


80 


10 


Ron 


•j 














80 


10/10 


800 


3/10 


240 


70 


9 


630 


2 


140 












70 


9/10 


630 


2/10 


140 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



PRODOCT 
QUALIFICATIONS 



VENDOR 



ER?C 



GRAND TOTAL, PRODUCT QUALIFICATIONS 
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GRAND TOTAL, VENDOR 



WEIGHT 
(W) 

80 



80 



70 



70 



TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 



720 



9/10 



720 



560 



8/10 



560 



SIRS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 



320 



4/iO 



4/10 



320 



280 



280 



220 



7 * 2 Relative Suitability of SIMS to the Junior High Schools 



The relative suitability of SIMS to the junior high schools was 
determined using the same procedure and the same percent emphasis 
distribution as was used ?\i the senior high school situation (see section 
5). The outcome of this procedure is shown in the table below. 



EVALUATION FACTOR 


EMPHASIS 
(%) 


RELATIVE PRODUCT SUITABILITY 


TSS 


SIRS 


PRODUCT SCOPE AND FUNCTION 
EASE OF USE 

TECHNICAL CONSIDERATIONS 
SUPPORT AND SERVICES 
PRODUCT QUALIFICATIONS 
VENDOR 


45 


34 


18 


1 0 


9 


5 


10 


10 


3 


15 


13 


3 


10 


9 


4 


10 


8 


4 








TOTALS 


100 


83 


37 



The following two tables parallel the simulations which were provided 
relative to the senior high school situation. Appendix 6 shows yet 
another simulation in which not all of the evaluation criteria were used 
and in which the actual criteria weighting factors were adjusted to the 
needs of a partirular user. This particular simulation has been 
performed using senior high school data but the principles involved are 
equally applicable to the junior high school data. 



2 ? t 

*•* C J. 



0 

ERJC 



(130) 



SIMULATION 1 - JUNIOR HIGH PERSPECTIVE 



EVALUATION FACTOR 


fTMPU AC , C 

("0 


RELATIVE PRODUCT SUITABILITY 


TSS 


SIRS 


PRODUCT SCOPE AND FUNCTION 
EASE OF USE 

TECHNICAL CONSIDERATIONS 
SUPPORT AND SERVICES 

PPnniTPT HTTAT tCTrATTHMC 

VENDOR 


55 


42 


22 


20 


18 


10 


5 


5 


1 


10 


9 


2 


J 


4 


2 


5 


4 


2 








TOTALS 


100 


82 


39 


SIMULATION 2 - JUNIOR HIGH PERSPECTIVE 


EVALUATION FACTOR 


EMPHASIS 
(%) 


RELATIVE PRODUCT SUITABILITY 


TSS 


SIRS 


PRODUCT SCOPE AND FUNCTION 
EASE OF USE 

TECHNICAL CONSIDERATIONS 
SUPPORT AI^D SERVICES 
PRODUCT QUALIFICATIONS 
VENDOR 


50 


38 


20 


20 


18 


10 


10 


10 


3 








20 


18 


8 














TOTALS 


100 


84 


41 
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8.0 COMMENTS AND CONCLUSIONS 



The major objective of this evaluation project was to comparatively evaluate 
microcomputer based School Information Management Systems and, in the process, 
to determine the viability of their use by schools. 

Three systems were evaluated against the same detailed set of criteria and in 
IBM Microcomputer environments. Initial experiences of the project team 
indicated that the application of microcomputers tc school information 
management was not well established. Software products wer3 not mature and 
the hardware environments in which they would run were not at all well 
defined. The School System by Columbia, for example, did not exist as a 
practical alternative when the project begar During the course of the 
project, this application of the technology matured considerably to the point 
that not only are such applications possible now but there are alternatives 
from which to choose. 

We are able to conclude from this project that at least one, and probably two, 
microcomputer based SIMS are available which allow a distributed approach to 
the school records, student scheduling, attendance and progress tracking and 
reporting functions. The results of this project further show that one 
system, notably, The School System by Columbia Computing Services, can 
effectively meet the needs of both the junior and senior high schools. This 
is a particular advantage to districts or ju "isdictious which seek to 
recommend and support a single alternative for all high schools. From the 
senior high school perspective, The School System scored seventy six of one 
hundred suitability points compared to sixty two of one hundred points awarded 
to Management Information Group's Student Information and Records System. The 
project showed the relative suitability of The School System to the needs and 
requirements of the junior high schools to be even more pronounced. 

The choice of systems, however, should be made only after very careful 
consideration. The software systems which were evaluated ranged in price from 
about four thousand to six and a half thousand dollars and are thus considered 
to be affordable. They can be effectively run on single user IBM 
microcomputers with hard disk drives which provide sufficient capacity to 
accommodate the needs o f even the largest senior high schools. The two 
microcomputers which wet i found to be most appropriate were the IBM PC/XT and 
the IBM PC/AT with the latter clearly being the preferred choice. A typical 
IBM PC/AT configuration (single user) with a printer will cost a user of the 
order of seven or eight thousand dollars. 

Those considering the implementation of one of the microcomputer based SIMS 
alternatives which were tested through this work should carefully examine the 
process for determining product suitability and re-apply it to the raw 
evaluation data from their particular perspective. Those who seek to identify 
another alternative are encouraged to apply the principles cf this process to 
the maximum extent possible. 

In closing, it is noted that the project reported on here is part of a more 
comprehensive evaluation of the distributed approach to school information 
management.. A further report will address the viability of a minicomputer 
based approach to school information management. 
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APPENDIX 1 
GENERAL QUESTIONNAIRE 



This document was distributed to schools for completion 
as an initial information gathering step in the process 
to develop evaluation and selection criteria for school 
information management systems. 
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EDMONTON PUBLIC SCHOOLS 
COMPUTERIZATION OF SCHOOL ADMINISTRATIVE/INFORMATION SYSTEMS 



GENERAL QUESTIONNAIRE 

Background 



The Distributed Systems Services Team has identified a short list of compu- 
ter software packages specifically designed for the day-to-day student 
administrative requirements of individual schools. In order to facilitate 
the selection of the most suitable software alternative, for the EPSD from 
a District-wide perspective, the attached questionnaire has been prepared 
with a view of determining the relative importance of the type of inform- 
ation, system functions and features needed by the school(s). In addition, 
personal interviews will be conducted with each participating school in 
order to determine each school's specific information requirements, review 
the type and detail of data needed by the school to streamline its oper- 
ations and identify any areas or concern. 



The questionnaire has been divided into two parts. Part 1 deals with the 
information ne.ds of a STUDENT ADMINISTRATIVE SYSTEM and Part 2 addresses 
otner information requirements that the school (s) may have. 



Part 1 - STUDENT ADMINISTRATIVE SYSTEM 



Each item is to be weighted in accordance to its relative importance to the 
specific institution completinq the questionnaire, usinq the followinq rat- 
ing scale. 



NONE - Not required. 
OPT - "Optional" 



a requirement not considered essential but 
for which preference may be qiven 



IMP - "Should" - a requirement having a signifi cant degree 

("Desireable" ) of importan ce to the objectives o? the 
("Important") Student Administrative/information System 



MUST - Mandatory 



a requirement that must be met in a sub- 
stantially unaltered form in order for the 
software package to meet the schools vital 
information needs. 



Part 2 - OTHER INFORMATION SYSTEMS 



Applications shtuld be ranked in accordance with the school's priority to 
computerize other areas of its operations. 
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NAME OF SCHOf (in ful I) _ _ 

<i'ectionnaire completed by: (Name) 

(Title) 



PART 1 

STUDENT ADMINISTRATION SYSTEM - INFORMATION NEEDS 

SECTION A - School records, student records, attendance recording/ 
reporting, student marking process and reporting 
requirements. 

General Overview of the System's Objectives 

A computerized student administrative system to resolve and streamline the collecting, 
transcribing, maintaining and reporting of student data. It is to maintain student relat- 
ed data, provide up-to-date information and prepare reports that are used by administra- 
tors, counsellors, instructors, students and parents. 

Info r matio n Need - Relative Rating Scale Legend : 

Relativ e Impor ta n c e 

Column Heading - NONE OPT IMP MUST 

Degree of importance - Not required Optional Important Mandatory 
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Application/Feature Description Relative Importance 



NONE OPT IMP MUST 



1 ) Registration/Enrol Iment 

-Enterinq a student into the school and 
creating the student record 

-Registration/Enrollment confirmation notice 

-Other information needs (specify): 



2) Student Records 

-Demographic data e.g. name and address, pro- 
qram, type of instruction, medical, class(es), 
timetable, medical, parents, etc. 

-History i.p. academic achievements, marks, 
course attempts, etc. 

-Student coding e.g. 

- school ID# 

- EPSD & Alerta student ID # 

-Bus Information e.q. bu^ pass number, pick- 
up and drop off points, driver name, bus 
routes etc. 

-Interface/integration with your school's 
accounting system (in future) 

-Other (specify) 



*1< £ 



2/ 
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Application/Feature Desc riptic Relativ e Importance 

NONE OPT IMP MUST 

3) Student Attendance 



-Indicate the frequency that attendance is/ 
should be taken in your school e.g. every 
period (by class) once per day, twice per 
day, at homeroom time, etc. 



-How often do you need attendance reports 
e.g. daily, weekly, bi-weekly, etc.? 



-How much detailed attendance history does 
your school require to keep "on-line" for 
parent, counsellor inquiries e.g. 5 days 
history, 6 days history etc.? 



-What types of attendance reports do you need? 
e.g. by student, student by cl ass/sub.iect , 
student by day, exception reports etc. and 
how frequently do you require each report? 



4) School Reports 

-Directories/class lists 
-Labels (mailing) 
-Student ID cards 

-Schedules (student, teachers, rooms) 
-Other reports (specify) 



9 
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Appl ication/Fpaturp Descrip tion __Rp l_ati v p J mport an nc e _ 

NONF OPT IMP MUST 



5) Instructor Records 

-Personal and demographic information 
-Courses tauqht 
-Areas of specialty 
-Certificate number 
-Other (specify) 



6) Student Marking Process 

-Comprehensive editing and validation of student 
marks prior to report card Preparation e.q. mark 
verification, identification of student with 
unassiqned marks Ptc. 

-Report card printing 

-Type of reports e.q. GPA's, honour lists, etc. 
(Please specify): 



- 'ther information needs (specify) 



-What is the maximum number of marks per ccrse 
maintained by your school for a student e.<*. 
4 mid-term marks, 2 exams and a final mark? 
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Application/Feature Description 



NONE 



Relative Importance 



OPT 



IMP 



MUST 



7) Student Exans 



-Exam timetable builder 
-Exam conflicts matrix 
-Exam schedules 
-Other (specify) 



8) Courses 

-Course nunber, short description, detailed 
description (for annual school handbook), 
credit values, prerequisites, etc. 

-Other information requirements (specify): 
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SECTION B - STUDENT SCHEDULING 



Course requests, prerequisite vesication, request confirmation, student curricular coun- 
selling, computerized scheduling, school start up registration, automatic generation of 
student fee ?ts and printing of individual timetables. 



THIS SECTION IS APPLICABLE TO HIGH SCHOOLS, 
JUNIOR HIGH SCHOOLS AND ELEMENTARY-JUNIOR 
HIGH SCHOOLS ONLY 
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SECTION B - STUDENT SCHEDULING 



Course requests, prerequisite verification, request confirmation, student curricular 
counselling, computerized scheduling, school start up registration, automatic generation 
of student fee sheets and printinq of individual timetables. 



Application/Feature Description Relative Importance 

NONE OPT IMP MUST 

1) Pre-scheduling 

-Comprehensive editing and validation of 
course requests e.g. prerequisite checkinq 
marks verification, identification of 
students with no requests, insufficient/ 
excessive credits requested 

-Preschedul inq reports e.g. course tally " 

list, exception reports (students missing 
manaatory/compul sory courses) 

-Scheduling conflicts matrix " 

-Other information needs (specify): 



-Other preschedul ing reports (specify): 



2) Master Schedule 



-Master timetable builder 



i) What course code would you prefer to 
use e.g. a school course code, EPSD 
course code or the Alberta course code 



ii) Please specify ALL of the scheduling 
units used by your school, e.g. semester 
full year, tri semester, six week section, 
quartermester, etc. 



ERLC 
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Application/Feature Description Relative Importanc e 



NONE OPT IMP MUST 



iii) Please specify the following: 

Rotation: 

Days per week: " 

Periods per week: ~ 

used in your school's master timetable. 



3) Student Scheduling 

-Completion of the student scheduling process 

before the summer break 
-Ability to preassign sections 
-Ability for your school to assign schedulinq 

priorities 

-Automatic scheduling of an individual student 

i.e. mid-term transfer pu^il 
-Ability to schedule groups of students 

i.e. unregistered last minute arrivals 
-Ability to 'UNSCHEDULE 1 * a student or qroup 

of students i.e. no shows, students that 

move away during summer etc. 
-Restart capabilities e.g. reset assignments 

for a student and/or course 
-Course sequencing 

-Course weighting i.e. ability of the computer- 
ized scheduler to distribute course loads evenly 
so that a student is not scheduled to take an 
overload of difficult courses in the first 
semester and a qroup of relatively easier 
courses during the second semester 

-Blocking 

-Class balancing 

-Semester balancing 

-Double room identity e.g. Physical Education 
all male/female class 

-Double room identity for mixed classes e.g. 
Home Economics and Industrial Arts 



i) What are your present scheduling priorities 
e.g. - lower grade students first and so 
on up to highest qrade? 



e.g. - single section courses before 
multiple section courses? 
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Application/Feature De scription Relative Importance 

NONE OPT IMP_ MUST 

e.g. - mandatory/compulsory courses first 
followed by student preferences 
followed by options/alternatives? 



OR indicate your priorities in the space 
below: 



-Ability to run schedules from more than one 
perspective e.g. single sections first then 
mandatory courses etc. and mandatory courses 
first and single sections last 

-Other information needs (specify): 



Reports 

-Student schedules 
-Multiple conflicts matrix 
-Partially scheduled students 
-Other (specify): 



School Start Up 

-Generation of fee sheets 

-Ability to schedule all new students (unexpect- 
ed enrollments) only i.e. the schedules for all 
previously registered students would not be 
affected 

-Preparation of timetables in grid format 
(students, teachers and rooms) 
-Class lists 
-Other (specify): 
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THE FOLLOWING ITEMS ARE PERCEIVED TO BE APPLICABLE 
TO SCHEDULING IN JUNIOR HIGH SCHOOLS ONLY 



App ■ 1 lie : at ion/Feature Description Relative Importance 

NONE OPT IMP MUST 

5) Special Scheduling Requirements 
of Junior High Schools 

-Blocking of course options 
OR 

Scheduling students requesting same group of 
options into the same class or homeroom 



-Blocking of 2-3 sections of the same course 
in same time block e.g. Math or Language Arts 



-Homeroom identity grouping for Languaqe Arts, 
Social Studies, Science, Math 



-Ability to handle option courses with varying 
lengths of instruction e.g. French as an option 
requires four periods per week whereas other 
options require three periods per week 



-Back to back time tabling for double classes 



-Ability to handle variable time slots by 
course subject e.g. six periods of Languaqe 
Arts, five periods of Math, four periods of 
Social Studies, etc. 



-Other requirements or unique characteristics 
associated with the scheduling process for 
your school 



Please specify any idiosyncracies in your 
schools allocation of subject time e.g. 
different/variable periods (standard period 
= 40 minutes, course x has a period of 
30 minutes, etc. ) 
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PART 2 - OTHER INFORMATION SYSTEMS 

Please rank the importance of each application in accordance with your 
schools priority to computerize other areas of its operations, e.g. 1, 2, ? 
etc., from most important to least important. If an application is not 
perceived to be a requirement indicate a priority of "O' 1 (zero) or "NIL". 

Impl ementation 

Application/System or Sub-system P riority 

Accounts Payable 
Accounts Receivable 
Budgeting 

Computer Assisted Instruction (CAI, CAl, CML) 

Cost Accounting 

Financial (General Ledqer and Financial Statements) _ 

- also indicate whether or not you reqrire ~~ 
commitments to be included i.e. encumberance 
accounting Yes or No ___„ 

Fixed Assets 

Inventory Control 

LiDrary Services 

Purchasing 

Word Processing 

Work Orders 

Other (Specify) 
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INTERVI Ef GUIDE AND DETAILED CHECKLIST 



This document was used to facilitate a follow-up 
interview with surveyed schools to clarify and confirm 
their responses to the general questionnaire. 
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EDMONTON PUBLIC SCHOOLS 



COMPUTERIZED INFORMATION SYSTEMS NEEDS OF INDIVIDUAL SCHOOLS 



INTERVIEW GUIDE AND DETAILED CHECKLIST 



SECTION A - School records, student records, attendance rerordinq/ 
reporting, student marking process and reporting 
requirements. 



Application/Feature Description Relative Importance 



NONE OPT IMP MUST 



1 ) Regi strati on/Enrol Iment 

Use questionnaire. 



2) Student Records 

-Personal /Demographic 

-Courtesy name 

-Academic 

-Activities 

-Medical 

-Program 

-Type of instruction 
-Timetables 
-Courses and classes 

-Student history to include all courses/marks 
while in the school 
OR 

Uoes the school want to include all marks the 
student has achieved while in a similar level 
of school e.g. High School, Grades 10-12; 
Junior High, Grades 7-9 etc. 
Specify level of detail needed below: 



-Complete history of each course that each 
student attempts, includinq the number of 
attempts 



-Parent data up to a maximum of 2 parents 
per student 
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Appl i catio n/Feature Descri ption ____ Relative Importance 



NONE OPT IMP MUST 



•Is a limit of 2 parents sufficient? 
Yes or No 



-Bus pass number 
-Bus route(s) 
-Driver name 

-Pick-up and drop off points 

-Student ID # (indicate whether the school 

has a preference for its own unique ID 

system or the EPSD ID #) 



-Multiple ID'S for cross referencing and 
interface with EPSD and Alberta 



3) Student Attendance 

Use questionnaire. 

4) School Reports 

Use questionnaire. 

5) Instructor Records 

Use questionnaire. 

6) Student Marking Process 

-Report cards prepared by school rather 

than ISB Yes or No _ 

If Yes indicate level of importance 

-Student marks proof listing for verification 
before production of report cards 

-Student transcripts 

7) Student Exaras 

Use questionnaire. Z'3 j 
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Application/Feature D es cription 



Relative Importance 



NONE OPT IMP MUST 

8) Courses 
-Term weight 

-Included/excluded from report card averaae — " 
-Pass/Fail mark 
-Other (specify): 



SECTION B - STUDENT SCHEDULING 



N.B. THIS SECTION SHOULD BE COMPLETED FOR HIGH SCHOOLS AND JUNIOR 
HIGH SCHOOLS ONLY 



Application/Feature Description Relative Importance 

1) Pre-scheduling 

-Student course/proqram/curriculum counselling 
list 



-Marks verification as part of prerequisite 
checking e.g. 49% in Math 10 is not acceptable 
for entry into Math 20 course but is acceptable 
for Math 23 

In this case should the student be advised 
of his/her options before the scheduling 
simulatir, i.e. repeat Math 10 or opt for 
Math 23? Yes or No ? 



-Ability for the individual student to 
identify his/her 

a) mandatory/compulsory courses 

b) preferred course requests 

c) preferred alternatives 



CONTINUED 



Application/Feature Description 



Rel . ive Importance 



NONE 



t T 



IMP 



MUST 



-Ability to conduct prerequisite checking for 
students from another school within the EPSD 

-Ability to handle co-requisites 

-Ability to add student records from another 
EPSD school into your microcomputer e.g. 
transfer student, graduate student from 
a feeder school etc. 



2) Master Schedule 

-Current Semester 
-Current Year 
-Future Semester(s) 
-Other (specify): 



3) Student Scheduling 

-Access to scheduling alorithim e.g. logic, 
parameters, scheduling resolutions, options etc, 

-"Teacher Link Courses" e.g. in the instance 
where a teacher is instructing English 10 
and Social 10, a common core of students 
should be scheduled to this teacher for 
both courses (subjects) 

-Arena scheduling 

-Student section selection (preference) 
-Student instructor selection (preference) 
-Reduced term requests i.e. scheduling a 
student into, say, the second semester of a 
full year English course in order to improve 
his/her qrade without repeating the first 
semester which he/she passed satisfactorily 

-Specific term requests e.g. Biology 10 in 
first semester and Biology 20 in the second 
semester 



CONTINUED 
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Application/Feature Description Relative Importanc e 



NONE. OPT JMP MUST 

-Other requirements for an in-house computer- 
ized scheduler: 

- use data from q jestionneire and interview 

4) School Start Up 

Use questionnaire. 



5) Special Scheduling Requirements 
of Junior High Schools 

Use questionnaire. 

ENSURE THAT THE JUNIOR HIGH SCHOOL IDENTIFIES 
ITS UNIQUE NEEDS ANn DEFINES ANY ITEMS OR 
AREAS THAT DIFFER FROM THE NORM. 



0 
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PAKT 2 - OTHER INFORMATION SYSTEMS 



AC COUNTS PAYABLE (A/P) 

1) Open item or balance forward 

Does the school issue its own A/P cheques' 

If Yes how many cheques does it issue per month on the avpr^qe 7 

3) What is the average jmber of General Ledger distri but .on> per vendor invoice? 

4) If the school has indicated that the computerization of its Accounts Payable applica 
tio-i is a need, obtain a general description of what the sciioo ixpects from an auto 
mated system e.g. type of reports, statistica 1 analysis, breakdown of A/P expenses 
(how?) etc. 



5) Should thr school's purchase orders be included *:i the A/P system to reflect com.nit- 
ments? 
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ACCOUNTS RECEIVABLE ( A/R) 

1) Open item or balance forward 



2) How many invoices does the school issue per month? 



3) Does the school issue monthly statements for unpaid accounts? 



4) Why does the schoo' want to automate its A/R application? 

e.g. expected rest' type and frequency of reports, revenue analysis, etc.? 



BUDGETING 



If computerization of General Ledger and Financial Statements are a need identified by the 
school suggest tr at the budgeting application should be included as an integral part of 
the former system. 



1) What information and/or statistical breakdowns do we need for budgeting e.g.: 
student count by category or program (ESL pupils, native children, etc.) 



-previous years financial statements by department, proqran, cost centre, etc. 
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FINANCIAL (GENERAL LEDGER AND FINANCIAL STATEMENTS) 

1) Should commitments be included in the schools financial reports i.e. encumberance 

accounting in order to ensure that the school knows where it stands n 'n relation to its 
budqetl 

For example: 

Total budget (actual expenditures + PO commitments) = the balance available in the 

budget 



2) Does + he school require any interface/integration between its financial and student 
administrative system? 



3) What type of G/L codinq structure does the school envision? 

e.g. EPSD G/L code 
or 

The schools own G/L cede 



4) How many G/L accounts does Lhe schuol now use? 
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5) What objectives ii the bChO): seeking through computeri zjitic < of its financial inform- 
ation i.e^ type and frequency of reports, budqet analysis etc. 

6) How many different fund sources does the school have? 
e.g. 

EPSD funds (from provincial and municipal taxes) 

TRIM funds (Text booL rental, fees and instructional materials) 

Special project funds derived from school initiatives i.e. car washes, 
bottle drive etc., for field trips (qlee club, band, soccer team) 

Other 

7) Does the school require separate financia' statempnts for pach fund it is responsible 
for? 



8) Are consolidated financial statements required by the school? 



9) What other financial information does the school need? 
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COMPUTER ASSISTED INSTRUCTIO N 

Obtain a general description of Vie schools needs and expectations in this cr*ea. 



Cost Accounting 

1) Could the schools requirements in tiiis area he ircluded in the qcneral ledger finan- 
cial statements. If not obtain a conceptual overview of the type of cost accounting 
information required by the school. 
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FIXED ASSET S 

1) What qeneral class of items does the school want to include in this application? 



2) Are the school's fixed assets currently tagged with a permanent identifier? 



3) Approximately how many items doec the school estimate it would include in its automat- 
ed fixed asset sysem? 



4) Obtain a brief conceptual overview of what the school expects from a fix id asset 
system. 



5) What type and frequency of reports does the school need ^ron. this system. 
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INVENTORY CONTROL 

1) Does the school have a central storaqp facility? 



2) What type(s) of inventory and how m?ny items, issues and receipts does the school wish 
to control? 

e.g. Automotive shop 

Wood shop 

Home Economics, etc. 



3) Does the school need to integrate its purchase orders with invpntory control? 



4) What does the school need in the way of an inventory control system 7 
Describe briefly. 
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LIBRARY SERVICES 

1) How many books does the school pstimate to have in its library? 



2) Computerized needs 



-Cross Reference by 



Author? 



Title? 



Publ isher? 



Subject? 



Key words 7 



-Checkout/Renewal 
-Returns 

-Overview notices/lists 

-Fines 

-Other 

3) Statistics e.g. usaqe? 

4) Obtain a general conceptual overview of the schools needs in this 



ERIC 



(159) 




- 9 - 

PURCHASING 

General requirements, volumes a.,-1 brief conceptual overview. 



WO RD PROCESSING 

Estimated volumes, frequencies 

Type of word processing needed i.e. 
personalized letters 
mass mail i ngs 
report^ 

qeneral correspondence 
Try to determine an estimate of the school's current work load. 
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WORK ORDERS 

Estimated Volumes 

How are they handled now? 

Are W/O's costeci out e.g. 

labour $ 
material $ 



Are W/O's inteqrated into the financial system 7 



General conceputal overview and dpscription of systen need 
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DETAILED SCORING COMPARISON FORM 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


PRODUCT 1: 


PRODUCT 2: 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


PRODUCT 3: 


SCORE WEIGHTED 
SCORE 
(S) (V X S) 


<%CORF WFrmrfn 

SCORE 
(S) (W X *) 


PRODUCT 
SCOPE & 
PUNCTIOH 

1 


SCHOOL RECORDS 

Pre-Registrat ion/Enrollment 

Create student record 

Registration confirmation notice 
Feeder school confirmation notice 

TOTAL Pre-Reglstratlon/Enrollaent 

Detailed Data Items 
Student information 
Instructor Information 
Course information 

TOTAL Detailed Data Iteas 
Reports/Inquiries 


15 






1 


3 








2 








20 








25 








5 








15, _ 








*5 








25 








TOTAL Reporta/Inquf rlea 
TOTAL SCHOOL RECORDS 
SCHEDULING 

Manual scheduling (Arena Scheduling) 
■ i 


25 









90 








7 
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or 



25 o 



1 

EVALUATION 


CRITERIA ITEMS 




Pre-scheduling 




Course Requests 




manual entry 
automated entry 




Edit and validation of course requests 




Pre-scheduling reports 




TOTAL Pre-Scheduling 




Master schedule builder 




Capability to buixd a master scheduler 

manually 

automatically 
Capability of handling a variety ^ f 
scheduling unit** 




User defined timetable rotation/tumble 
Flexible number of periods per day 
Capability to specify exclusive male or 
fimale sections 

Capability to maintain current anu future 
year/secester master schedules 




TOTAL Master Schedule Builder 




Scheduling Proces. 




User defined scheduling sequence 




Unscheduling of no-shows/ wi thdrawals 
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WEICHT 
(W) 


PRODUCT 1 : 

SCORE W£IGHTE0 
SCORE 
(S> (W X S) 


3C0RE WEIGHTED 
SCORE 
(S> (W X S) 


P.xODUCT 3 : 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


5 








9 








7 









9 









30 








6 










9 








9 








10 








10 




-*— — 


■ 


5 








8 








57 

















5 
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EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 
(W) 


PRODUCT I; 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


PRODUCT 2: 

SCORE WEICHTED 
SCORE 
(S) (W X S) 


PRODUCT 3: 

SCORE WEICHTED 
SCORE 
(S) (W X S) 




Scheduling of Individual student or small 

groups of students 

Capability to reset all students or 

partially scheduled students 

Capability to lock scheduling assignments 

for all students or a group of students 

Restart capability 

Course weighting/semester balancing (ensure 
even course load for studentc) 
Blocking ol courses 
Section balancing 
Class balancing (m*les-f emales) 
Capability to k»ep scheduling op^n after 
school start while starting to us* the 
attendance module 

TOTAL Scheduling Process 

Scheduling Reports/Inquiries 


6 








8 








8 








3 










8 


- 






7 








8 









k 









9 








77 








1C 








Junior High Scheduling Requirements 

Homeroom grouping for core subjects 
Capability of scneduling any course iu my 
combination and number o r time periods 

TTT^L SCHEDULING 

STUDENT ATTENDANCE 

En.ry of Attendance Data 

manual entry 
automated entry 


9 









\0 








200 








5 







_ 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



Multiple user-defined absence types 

Capability to record attendance data at 
various intervels 

Attendance bistcry 

At tendance reports/inquiries 

TOTAL ATTENDANCE 

STUDENT MARKS 
Entry of marks Hata 



aanu.il 
automated 

Marks data 

Student Exams 

Exam timetable builder 
Exam Reports/Inquiries 

Reports/Inquiries 

TOTAL STUDENT MARKS 



WEIGHT 

(w? 


PRODUCT 1 : 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


PRODUCT 2: 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


PRODUCT 3: 

SCORE WEIGHTED 
SCORE 
(S; (W X S) 


8 








10 








8 








10 









50 








5 








9 









10 









6 








10 






i 


40 


. — 




\ 









I 
I 

j 
i 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



UTILITY FUi CTIONS 

Backup/Restore 
Security/Controls 
TOTAL UTILITY FUNCTIONS 



GRAND TOTAL, PRODUCT SCOPE AND FUNCTION 



EASE OF i 
USE 



TECHNICAL 
CONSIDERATION 



SUPPORT & 
SERVICES 



GRAND TOTAL, EASE OF USE 



GRAND TOTAL, TECHNICAL CONSIDERATIONS 



GRAND TOTAL, SUPPORT & SERVICES 
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WEIGHT 
(W) 



PRODUCT 1 
SCORE WEIGHTED 
SCORE 
(O (W x S) 



PRODUCT 2 



SCORE 



(S) 



WEICHTED 

SCORE 
(W X S) 



PRODUCT 3 



SCORE 



(S) 



WEIGHTED 

SCORE 
(W X S) 



8 



400 



60 



60 



80 



80 



70 



70 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 
(W) 



PRODUCT 
QUALIFICATIONS 



GRAND TOTAL, PRODUCT QUALIFICATIONS 



ON 

00 



VENDOR 



GRAND TOTAL, VENDOR 
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WEICHT 
(S) 

80 



80 



PRODUCT i: 



SCORE WEIGHTED 

SCORE 
(W X S) (S) 



PRODUCT 2: 



SCORE WEIGHTED 

SCORE 
(W X S) (S) 



PRODUCT 3: 



SCORE 
SCORE 
X S) 



WEICHTED 



70 



70 




bo 



APPENL7X 4 



SIRS USKR GROUP MEMBER LIST 



F e b r unry 4 _ 1_9 8 5 



School 



Contact 



Phone 



Foothills Composite High School 
Okotoks, Alberta 
TOL 1T0 



Torry Storch 



938-6116 



Grande Cache Community High School 
Box 599 

Grande Cache, Alberta 
TOT, 0Y0 



Doug Perras 



827-3502 



Paul Kane High School 
12 Cunningham Road 
St . Albert , Al ber ta 
T8N 2£9 



Donna Powell 



459-4405 



Redvater School 
Box 790 

Redvater, Alberta 
TOA 2W0 



Henry F I eg e 



942-3625 



Richard F. Staples High School 
Box 369 

Veatlock, Alberta 
TOG 2L0 



Art At t ken 



349-4454 



R~>cky Mountain House Jr/Sr High School 
Box 1840 

Rocky Mountain House, Alberta 
TOM 1T0 



Forest Pi rd 



845-3 711 



Sturgeon Composite High School 

Natnao , Alberta 
TOA 2N0 



Ca 1 Co s h 



973-3301 
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MANAGEMENT INFORMATION OROUP 



Appendix 5 Recent Product Developments 



Since completing the formal evaluations of the three microcomputer based 
packages in February 1985, there have been a number of product announcements 
and system enhancements. The following sub-sections list important 
developments that are known at the time of writing. 



MIG SIRS 



Core subject blocking: 



Up to 9 classes can be blocked together to 
provide core subj ect sections for groups of 
students. For example, in a Junior High School, 
subjects such as Physical Education and 
Mathematics are compulsory and have to be blocked 
together for different homeroom groups. 



Adding speed and 
function to the 
scheduler 



The development plans call for an increase in the 
speed and f unction of the Srhpdnlpr by rp ?rd1ng 
parts of the software in "C" and reint cting 
scheduling parameters . 



T nterface with the NCS 
SENTRY 3000 scanner: 



Software is being developed to handle automatic 
scanning of course requests and other standard 
forms. This is very important for a large 
school. Edmonton Public School District is 
already using the SENTRY 3000 scanner with 
another software package. 



Interface with the 
"Surveyor" auto dialing 



This development will provide student home 
telephone numbers to the "Surveyor" system used 
to call parents when stud^^s are absent ^r late, 



COMPUTERLIB CEMAS 



Computerlib has been informed of the evaluation vork and has been asked to 
send information on recent developments. At this point in time wf have 
not received replies to our requests. 



COLUMBIA TSS 



Multi-user Facilities: The School System supports 2 users on an IBM XT 

and up to 4 users on an IBM AT us^ng the Multi- 
link software package and suitable RAM wemory 
boards . 



File Builder: 



The Fi le Bui lder produces sequential tt\t output 
files from the database. These can be used for 
data transfer to other computer systems. 



Prepare to Schedule 
New Year : 



A.ter running the process, . eports can be printed 
for the current scheduling year from the Scudent 
Scheduling system. The current and new school 
years can now be usee concurrently and students 
can be enrolled. 
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Mark Averages: 



These are now calculated using the "include in 
GPA" indicator in the Marks Directory. 



Report Reprinting: Reports can now je reprinted after pap?r jams or 

user requests for a second copy of the report . 

Academic Hiscory: A simple academic history facility has been 

provided but has not yet been tested by thp 
Distributed Systems Team. 
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Appendix 6 Relative Suitability of SIMS - Full Re-simulation 



As has been previously stated, the evaluation process is extremely flexible 
allowing a single ucer or group of useis to apply it from tt^ir particular 
needs/requirements perspective. The purpose of this appendix is to 
demonstrate this flexibility. 

In all previous simulations (in the main body of this report), the full set of 
evaluation criteria was used and critiera weighting factors were kept constant 
- only the percent amphasis distribution was changed. 

The following example assumes that products do not need to be evaluated 
against the complete set of criteria and further, that the weighting factors 
associated with these criteria which will be used should be changed. 

For th- ourpose of this example, the Detailed Scoring Comparison Form has been 
used to represent the data and to illustrate the changes which have been made. 
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TSS 




SIRS 


! 

CEMAS | 


EVALUATION 
FACTOR 


CRITERIA ITEMS 


WEIGHT 


SCORE 

(s; 


WEIGHTED 
SCORE 

(w X s) 


SCORE 

(s) 


WEIGHTED 

SCORE 
(W X S) 


SCORE 
( O 


1 

WEIGHTFD i 
SCORE ! 

f U Y \ 


PRODUCT 
SCOPE & 
FUNCTION 


SCHOOL RECORDS 

P re-Reg i st rat ion/Enrol imer.t 














i 
I 
1 

i 




Create student record 


0 


6 


0 


8 


0 


7 


0 


I 


Registration confirmation notice 
Feeder school confirmation notice 


i 0 


5 


0 


2 


0 


0 


0 




0 


2 


0 


0_ 


0 




U 




TOTAL Pre-Pegistrat ion/Enrol lta^n I 


0 




0 


i0/30 


0 


7 


0 i 




Detailed Data Items 


















Student information 


25 


a 

o 


zuu 


Q 

o 


zOU 


t% 
y 


225 




Instructor Information 


5 


3 


15 


7 


35 


•> 


1 1 




Course information 


15 


_8_ 


120 


8 


PO 


i 

l 


i j 




TOTVL Detailed Dp La Items 


45 


19/10 


335 




J J J 


12/30 


250 




Reports /Inquiries 


0 


u 


o 
u 


0 


u 


1 


0 




TOTAL *'irjrts/lnquiries 


0 


6/iO 


0 


6/10 


o 1 


1/10 


0 




TOTAL SCHCOL RECORDS 






J J D 






20/70 


250 




SCHEDULING 


















Menial scheduling (Arena Scr^dultng) 


0 




O 


9 


0 


8 


0 
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EVALUATION 
FACTOR 
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CRITERIA ITEMS 



Pre-schedullng 

Course Requeues 

manual <?ntry 
automated entry 

Edit and validation of course requests 

Pre-schec uling reports 

TOTAL Pre-Scheduling 

Master schecule but 1 de r 

Capability to build r master scheduler 

manually 

automatical ly 
Capability of handling a variety of 
scheduling uni ts 

User defined timetable rotation/tumble 
Flexible number of periods per day 
Capability to specify exclusive male or 
female sections 

Capability to nwiintai'i current and future 
year/semester master schedules 

TOTAL Master Schedule Builder 

Sch e duli -n* psb 

User de i i scheduling sequence 

Unfichedul Ing of no-shows/wt thdrawals 



WETOWT 



SIRS 



CFMAS 



°.CORE WIGHTE'J, F "ORE WEIGHTED j ST»RE 

SCORE I 



(S) 



SCORE 



(S) 



vW X S) 



r o 



weighted 
scope: 
(W X s) 



0_ 
30 



4 5 



32/40 15, 



30 



o" 

10 



10 



0 


4 


5 


3 


70 


4 1/70 


0 


5_ 


o 


9 







0 

J00 
300 



7 


35 


s 4 


20 


4 


0 


! 0 


n~ 


9 


63 


i 


21 


9 


81 


2 


18 


29/40 




9/40 


59 


7 


35 




40 


0 


0 


L _ 


0 


8 


u 


0 


0 


8 240 


0 


0 


8 


240 


4 


'20 


8 


0 


6 


0 


10 


50 


1 


5 


49/70 


565 


20/70 


165 



f3 



EVALUATION 
FACTOR 



CRTTERI A [TEMS 



Scheduling of individual student or small 
groups of students 

Capability to r-set all students or 
partially scheduled students 
Capability to lock scheduling assignments 
for all students or a group of students 
Restart capabilily 

Course weighting/semester balancing (ensure 
even course load for students) 
Blocking of courres 
Section balancing 
Class balancing (mal es-females) 
Capability to keep scheduling open after 
school start while starting to use the 
attendance module 

TOTAL Scheduling Process 

Scheduling Reports/ T nquiries 



Junior High Scheduling Requirements 

Homeroom grouping for core subjects 
Capability of scheduling any course in any 
combination and number of time periods 



TOTAL SCHEDULING 





STUDENT ATTENDANCE 




Entry of Attendance Data 




.nanual entry 




automated entry 
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WEIGHT 

(w) 


TSS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


STRS 

SCORE WE* HTED 
SCORE 
(S) (W X S) 


CEMAS 

SCORE WEIGHTED 
SCORE 
(S) (W X S) 


0 


8 0 


0 0 


7 0 


10 


1 

10 100 


0 0 


5 50 


5 


5 25 


0 0 


7 35 


0 


o c 


0 0 


1 0 


30 


7 210 


7 210 


A 120 


20 


9 180 


8 160 


0 0 


0 


8 0 


9 0 


0 0 


0 


7 0 


8 0 


2 0 


0 


9 0 


0 0 


0 0 


65 


77/110 SIS 


"*o/ III/ J /u 




10 


1 70 


9 90 


0 0 



















166 


166/240 1399 


144/240 


73/240 429 


5 


7 IS 






0 


8 " 


0 0 


0 0 
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TSS 


SIRS 


-7 

CEMAS ! 


FVAf 11ATTON 
FACTOR 


PPT TFD T A 


T TFMC 
1 1 trio 


r 1 17 tout 

Wh IGHT 
(W) 


SCORE 
(S) 


WEIGHTED 

SCORE 
(W X s) 


SCORE 


WEIGHTED 
SCORE 

k W A b ) 


SCORE 
\S) 


i 

WEIGHTED | 
SCOPE j 
(W X S) ' 

j 




Multiple use^-def lned 


absence types 


5 


9 


45 


8 


AO 


0 


0 : 




Capability to record attendance data at 
var'ous Intervals 


20 


9 


180 


9 


180 


6 


i 

120 




Attendance history 




10 


9 


90 


7 


70 


3 


30 




Attendance reports/in^alrles 


20 


6 


120 


7 


140 


2 


40 1 




TOTAL ATTENDANCE 




60 


48/60 


470 


40/60 


475 


17/60 


• i 

220 J 




STUDENT MARKS 




















Entry of msrks data 




















manual 




0 


8 


0 


8 


0 


u 


o 




automateo 




0 


8 


0 


0 


0 


0 


0 




Marks data 




0 


8 


o 


8 


0 


8 


o ; 




Student Exams 




0 


u 


0 


0 


0 


3 


o ; 




Exam timetable builder 
Exam Reports/Inquiries 
















i 

! 




Reports/ Inquiries 




0 


6 


0 


8 


0 


0 


1 

0 1 




TOTAL STUDENT HARKS 




0 


30/50 


0 


24/50 


0 


15/50 


0 


1 















ERIC 



EVALUATION 
FACTOR 



CRITERIA ITEMS 



UTILITY UNCTIONS 
SacVup/Res tore 
Security /Controls 
TOTAL UTILITY FUWCTiOMS 



EASE OF 
USE 



GRAND TOTAL, PRODUCT SCO'F AND PUNCTION 



GRAND TOTAL , EASE OF USB 



TECHNICAL 
CONSIDERATION 



GRAND TOTAL, TECHNICAL CONSIDERATIONS 



SUPPORT & 
SERVICES 



GRAHD TOTAL t SUPPORT » SERVICES 
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TSS 

WEIGHT SCORE WEIGHTED 
SCORE 

(W) (S) (W X S) 



sns 

SCORE WKICHTfclD 
SCORE 
(S) (W x :) 



96 



20 



291 



60 



60 



80 



80 



70 



70 



8/20 96 



2 90/440jj 230 0 



8 480 



8/10 



480 



720 



9/10 



720 



8 560 



8/10 I 560 
1 i 



8 96 



2 


16 


10/20 


112 




257/440 


2146 


6 


360 




[ 6/10j 


360 1 
i 


6 


480 




6/10 


480 


7 


490 




7/10 


1 490 



CEMAS 



SCORE 
(S) 



WEIGHTED 

SCORE 
(W X S) 



48 



6 


48 


10/20 


96 




135/440 


995 


3 


180 




3/10 1 
i 


180 




160 




2/:o| 


|l60 ] 


n 


0 




^/10 


0 
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EVALUATION 
FACTOR 



CRITERIA ITEMS 



PRODUCT 
QUALIFICATIONS 



VENDOR 



GRAND TOTAL , PRODUCT QUALIFICATIONS 



GRANT TOTAL, VENDOR 



ERIC 



TSS 

WEIGHT SCORE WFlCHTED 
SCOPE 

(W) (S) (W X S) 



SIRS 

SCOkE weighted 
SCORE 
(S) (W X S) 



CEMAS 
SCORE WEIGHTED 



(s) 


SCORE 
(W X S) 


0 


n 




! 

\ 
i 

! 


0/10 J 




i 


1 


o 1 



80 



9/10 



720 



720 



A 80 



6/10 


J 480 


■ - 

7 


0 



1 I I 0 ' i 

I I 



To remove a particular criteria (or set thereof) from consi jeration , the user 
simply sets the associated weighting factor to zero, This has the effect 
setting weighted scores o zero. The reader should nute these changes by 
comparing the tables in this appendix to the tables shown in section 5.1 . In 
this example, it can be seen that all weights associated with the SCHOOL 
RECORDS section have been set to zero, thus defining this particular feature 
to be irrelevant in determining product suitability. Similarly, the weighting 
factors associated with automated data entry within the context of scheduling, 
k-s also set to zero, thus implying that this particular feature is also 
irrelevant for evaluation purposes. Having eliminated certain criteria from 
consideration in this way, the relative importance of those which remain has 
been changed through adjustments to the weighting factors. For example, with 
the Pre-scheduling section, the reader will see that we have chosen to 
increase the importance of the user defined timetable rotation/tumble feature 
by changing its weighting factor from an (original) 10 to 30. Many other such 
changes have been made. Having done this, weighted scores have been re- 
calculated as shown and all totals appropriately adjusted. A summary of the 
important totals by major evaluation factors is shown in the table below 
together with the new ratios of weighted scores to maximum weighted scores. 









uivi Mr ru 

MAXIMUM 


(WEIGHTED SCORE; 


PRODUCT 


EVALUATION FACTOR 


WEIGHTED SCORE 


WEIGHTED SCORE 


(MAX. WEIGHTED SCORE) 


TSS 


PRODUCT SCOPE AND FJNCTION 


2300 


2910 


0.79 




EASE OF USE 


480 


600 


0.80 




TECHNICAL CONSIDERATIONS 


720 


800 


0.90 




SUPPORT AND SERVICES 


560 


700 


0.80 




PRODUCT QUALIFICATIONS 


720 


300 


0.90 




VENDOR 


0 


0 


0.00 


SIRS 


PRODUCT SCOPE AND FUNCTION 


2146 


2910 


0.74 




, EASE OF USE 


360 


600 


0.60 




'TECHNICAL CONSIDERATIONS 


480 


800 


0.60 




SUPPORT AND SERVICES 


490 


700 


0.70 




PRODUCT QUALIFICATIONS 


480 


800 


0.60 




VENDOR 


0 


0 


0.00 


CEMAS 


PRODUCT SCOPE AND FUNCTION 


995 


2910 


0.34 




EASE OF USfc 


180 


600 


0.30 




TECHNICAL CONSIDERATIONS 


160 


800 


0.20 




SUPPORT AND SERVICES 


0 


700 


0.00 




PRODUCT QUALIFICATIONS 


0 


800 


0.00 




VENDOR 


0 


0 


0.00 



Relative suitabilities are now c^culated in exactly the same way as before 
according to 



Relative Suitability = % Emphasis X ( weighted score ) 

(max weighted scorj) 

and are shown in the table following 



ERJC 



<179)2££ 



EVALUATION FACTOR 


| EMPHASIS 
(X) 


RELATIVE PRODUCT SUITABILITY 


TSS 


SIRS 


CEMA5 


PRODUCT SCOPE AND FUNCTION 
EASE OF USE 

TECHNICAL CONSIDERATIONS 
SUPPORT AND SERVICES 
PRODUCT QUALIFICATIONS 
VENDOR 


50 


J7 


37 


17 


15 


1 L 


Q 


c 
J 


0 


0 


0 


0 


30 


24 


?] 


n 


5 


4 


3 


0 


G 


0 


0 


0 










TOTALS 


100 


79 


70 


22 



A different percent emphasis distribution was used here which effectively 
removed Vendor and Technical Considerations from the evaluation. It is worth 
noting that removal of an entite major evaluation factor from consideration 
can be achieved by either setting all associated individual weighting factors 
to zero or by setting the percent emphasis to zero for that particular major 
evaluation factor to zero. 

The results of the above application of the process on a nartower set of 
criteria can be compared with the table in section 5.2 of this report. It is 
important to note that this application of the process is based on a purely 
hypothetical user perception and w«.s presented tor illustration purposes only. 

The final and most important point which must be made here Is that, while 
weighting factors and percent emphases were changed to reflect user 
perspective, the actual raw scores remained unchanged. Raw scores reflect the 
results of hands-on testing and evaluation by the evaluation team whereas 
weights and emphasis are at the discretion of the user and reflect his/her 
biases • 
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